learn deep. ship safe.Practice testsFollow ByteMunSubscribe

Concept

Transformers

The attention-based neural network architecture behind modern language models.

Concept map
Learn firstDeep learning
Transformers
Then learnLLMs

What should you learn before Transformers?

  1. Optimization and gradient descentAdjusting model parameters step by step to reduce a loss.
  2. Supervised learningLearning from labelled examples to predict a label or value.
  3. Neural networksLayers of weighted connections trained by backpropagation to learn patterns.
  4. Deep learningMachine learning with many-layered neural networks.

What to learn after Transformers

Connections to other concepts

Read and watch: Transformers explained

Practise Transformers

All topics