Technique
LLM evaluation
Testing language-model output with test sets, rubrics and model-graded checks.
LLM evaluation
What should you learn before LLM evaluation?
- Deep learningMachine learning with many-layered neural networks.
- Feature engineeringTurning raw data into inputs a model can learn from: scaling, encoding and new features.
- Machine learningBuilding systems that learn patterns from data instead of following hand-written rules.
- Supervised learningLearning from labelled examples to predict a label or value.
- Tokens and tokenizationSplitting text into the units a language model reads and is billed by.
- TransformersThe attention-based neural network architecture behind modern language models.
- Large language models (LLMs)Transformer models trained on large amounts of text to predict the next token.
- Model evaluation and metricsMeasuring how well a model works: train/test splits, accuracy, precision, recall, F1 and more.
Practise LLM evaluation
- Intermediate Generative AI Interview Questions3 questions on LLM evaluation
- Advanced Generative AI Interview Questions2 questions on LLM evaluation