1.1.4
What is autoregressive generation?
What is it?
Autoregressive generation is a process in which a model generates a sequence one token at a time. Each newly generated token is added to the existing sequence and becomes part of the context used to predict the next token. This continues until the model reaches a stopping condition.
Key points
1. The model starts with an input sequence.
2. It predicts and generates the next token.
3. The generated token is added to the sequence.
4. The updated sequence is used for the next prediction.
5. The process repeats until generation stops.
6. Previous generated tokens influence subsequent predictions.
Example: Building a sentence step by step
Starting prompt:
"AI can help"
A possible generation sequence:
Step 1: AI can help businesses
Step 2: AI can help businesses automate
Step 3: AI can help businesses automate repetitive tasks.
Each new token or token group becomes part of the context for subsequent generation.
Real-world application
Autoregressive generation is widely used in conversational AI, text generation, coding assistants and systems that produce responses progressively.
Key takeaway
Autoregressive generation builds text sequentially. Each new token depends on the context created by the preceding tokens.