single mention · software implementation
Separation of prompt encoding from inference
The software keeps prompt formatting separate from inference so formatting does not depend on PyTorch.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See software implementation for the methods filed beside it.
Also called separate encoding/ and inference/.
Filed under software implementation