learn deep. ship safe.Practice testsFollow ByteMunSubscribe

Technique

Knowledge-augmented generation (KAG)

Answering from a knowledge graph of entities and relations, for multi-step reasoning.

Concept map
Learn firstRAG
KAGCompared withCAGRAG

What should you learn before KAG?

  1. ChunkingSplitting documents into passages sized for embedding and retrieval.
  2. EmbeddingsLists of numbers that place similar meanings close together, used for search and clustering.
  3. Tokens and tokenizationSplitting text into the units a language model reads and is billed by.
  4. TransformersThe attention-based neural network architecture behind modern language models.
  5. Large language models (LLMs)Transformer models trained on large amounts of text to predict the next token.
  6. Vector search and retrievalFinding the stored embeddings closest to a query, usually with an approximate index.
  7. Retrieval-augmented generation (RAG)Fetching relevant passages from your documents and giving them to the model with the question.

Connections to other concepts

Compared with
CAGRAG

Read and watch: KAG explained

All topics