1.3.10
Model Size Reference
What is it?
The size of a language model is often described by its number of parameters. A parameter is a learned numerical value in the model. Parameter count is an important reference, but it does not alone determine capability, quality, speed or cost. Architecture, training data, training methods and inference setup also matter.
Key points
1. Parameters are learned numerical values.
2. Model size is commonly expressed in millions or billions of parameters.
3. Dense models use most of their parameters for each token computation.
4. Mixture-of-Experts models may activate only selected experts for a token.
5. Larger parameter counts usually require more memory.
6. Parameter count alone is not a complete measure of model capability.
Example: Understanding model size
A model described as 7B has approximately seven billion parameters.
A 70B model has approximately seventy billion parameters.
This does not mean the 70B model is automatically ten times better. Performance also depends on data, architecture, training and inference configuration.
Real-world application
Model size references help engineers estimate hardware requirements, memory needs, deployment costs and inference capacity.
Key takeaway
Parameter count is a useful model-size measure, but it should not be treated as a complete measure of intelligence or performance.