1.1.2
How does an LLM generate text?
What is it?
An LLM generates text by processing the input prompt and predicting what tokens should come next. It first converts the input into tokens, processes them through its neural network and calculates possible next tokens. It then selects a token and continues the process until the response is complete or a stopping condition is reached.
Key points
1. The user provides an input prompt.
2. The input is converted into tokens.
3. The model processes the token sequence.
4. It calculates probabilities for possible next tokens.
5. A token is selected using the decoding strategy.
6. The process repeats to generate the response.
Example: Generating a sentence
Input prompt:
"The sun rises in the"
The model processes the text and may predict the next token as "east".
The sequence becomes:
"The sun rises in the east."
The model can then continue generating more text if required.
Real-world application
This process powers conversational AI, automated email drafting, question answering, text completion and AI writing assistants.
Key takeaway
An LLM generates text step by step by repeatedly predicting and selecting the next token.