learn deep. ship safe.Practice testsFollow ByteMunSubscribe

Technique

LLM evaluation

Testing language-model output with test sets, rubrics and model-graded checks.

Concept map
LLM evaluation

What should you learn before LLM evaluation?

  1. Deep learningMachine learning with many-layered neural networks.
  2. Feature engineeringTurning raw data into inputs a model can learn from: scaling, encoding and new features.
  3. Machine learningBuilding systems that learn patterns from data instead of following hand-written rules.
  4. Supervised learningLearning from labelled examples to predict a label or value.
  5. Tokens and tokenizationSplitting text into the units a language model reads and is billed by.
  6. TransformersThe attention-based neural network architecture behind modern language models.
  7. Large language models (LLMs)Transformer models trained on large amounts of text to predict the next token.
  8. Model evaluation and metricsMeasuring how well a model works: train/test splits, accuracy, precision, recall, F1 and more.

Practise LLM evaluation

All topics