Concept
Small language models (SLMs)
Compact language models that are cheaper and faster, often fine-tuned for one job.
What should you learn before SLMs?
- Deep learningMachine learning with many-layered neural networks.
- Tokens and tokenizationSplitting text into the units a language model reads and is billed by.
- TransformersThe attention-based neural network architecture behind modern language models.
- Large language models (LLMs)Transformer models trained on large amounts of text to predict the next token.
Connections to other concepts
- Compared with
- LLMs
- Related
- Fine-tuning