Concept
Transformers
The attention-based neural network architecture behind modern language models.
What should you learn before Transformers?
- Optimization and gradient descentAdjusting model parameters step by step to reduce a loss.
- Supervised learningLearning from labelled examples to predict a label or value.
- Neural networksLayers of weighted connections trained by backpropagation to learn patterns.
- Deep learningMachine learning with many-layered neural networks.
What to learn after Transformers
- Large language models (LLMs)LLMs are large transformer models.
Connections to other concepts
- Related
- Hugging Face
Read and watch: Transformers explained
Practise Transformers
- Advanced Generative AI Interview Questions6 questions on Transformers