learn deep. ship safe.Practice testsFollow ByteMunSubscribe

Technique

Vector search and retrieval

Finding the stored embeddings closest to a query, usually with an approximate index.

Vector search and retrieval explained

Vector search finds the stored items whose meaning is closest to a query. Text, images or code are turned into embeddings, lists of numbers where similar meanings sit close together, and the search returns the nearest vectors by cosine similarity or dot product. It is the retrieval step behind semantic search, recommendations and retrieval-augmented generation (RAG).

How vector search works

  1. Chunk the documents into passages a few hundred tokens long.
  2. Embed each chunk with an embedding model and store the vector with its text and metadata.
  3. Embed the query with the same model.
  4. Find the nearest vectors and return their chunks, often filtered by metadata such as product or date.

Exact versus approximate search

Comparing the query with every stored vector is exact but slow at scale. Approximate nearest neighbour (ANN) indexes trade a little recall for a lot of speed. HNSW builds a layered graph you walk towards the closest points; IVF clusters vectors and searches only the nearest clusters; product quantization compresses vectors so more fit in memory. Each has settings that move the balance between recall, speed and memory.

Making retrieval accurate

  • Hybrid search: combine vector similarity with keyword search (BM25) so exact names, error codes and IDs are not missed.
  • Reranking: retrieve a few dozen candidates quickly, then reorder them with a slower, more precise model.
  • Good chunking: chunks that are too large blur the meaning; too small and they lose context. Overlap helps.
  • Measure it: build a set of real queries with known good answers and track recall at k.

Common mistakes

  • Using different embedding models for documents and queries.
  • Relying on vectors alone for queries full of identifiers or rare terms.
  • Never re-embedding after changing the model or the chunking.

What interviewers ask about vector search

  • "Cosine similarity or dot product?" They rank the same when vectors are normalised; cosine ignores length.
  • "Why approximate search?" Exact search grows linearly with the data; ANN keeps latency low at millions of vectors.
  • "Your RAG answers miss obvious documents. What do you check?" Chunking, hybrid search, the number of results retrieved and reranking.
Concept map
Vector search and retrieval
Then learnRAG

What should you learn before Vector search and retrieval?

  1. ChunkingSplitting documents into passages sized for embedding and retrieval.
  2. EmbeddingsLists of numbers that place similar meanings close together, used for search and clustering.

What to learn after Vector search and retrieval

Read and watch: Vector search and retrieval explained

Practise Vector search and retrieval

All topics