learn deep. ship safe.Practice testsFollow ByteMunSubscribe

Technique

Cache-augmented generation (CAG)

Loading a fixed knowledge set into the model's context and cache instead of retrieving per question.

Concept map
CAGCompared withKAGRAG

What should you learn before CAG?

  1. ChunkingSplitting documents into passages sized for embedding and retrieval.
  2. EmbeddingsLists of numbers that place similar meanings close together, used for search and clustering.
  3. Large language models (LLMs)Transformer models trained on large amounts of text to predict the next token.
  4. Tokens and tokenizationSplitting text into the units a language model reads and is billed by.
  5. TransformersThe attention-based neural network architecture behind modern language models.
  6. Prompt engineeringWriting instructions and examples that get reliable output from a model.
  7. Vector search and retrievalFinding the stored embeddings closest to a query, usually with an approximate index.
  8. Context engineeringChoosing what goes into a model's context window and in what order.
  9. Retrieval-augmented generation (RAG)Fetching relevant passages from your documents and giving them to the model with the question.

Connections to other concepts

Compared with
KAGRAG

Read and watch: CAG explained

All topics