Technique
Vector search and retrieval
Finding the stored embeddings closest to a query, usually with an approximate index.
Vector search and retrieval explained
Vector search finds the stored items whose meaning is closest to a query. Text, images or code are turned into embeddings, lists of numbers where similar meanings sit close together, and the search returns the nearest vectors by cosine similarity or dot product. It is the retrieval step behind semantic search, recommendations and retrieval-augmented generation (RAG).
How vector search works
- Chunk the documents into passages a few hundred tokens long.
- Embed each chunk with an embedding model and store the vector with its text and metadata.
- Embed the query with the same model.
- Find the nearest vectors and return their chunks, often filtered by metadata such as product or date.
Exact versus approximate search
Comparing the query with every stored vector is exact but slow at scale. Approximate nearest neighbour (ANN) indexes trade a little recall for a lot of speed. HNSW builds a layered graph you walk towards the closest points; IVF clusters vectors and searches only the nearest clusters; product quantization compresses vectors so more fit in memory. Each has settings that move the balance between recall, speed and memory.
Making retrieval accurate
- Hybrid search: combine vector similarity with keyword search (BM25) so exact names, error codes and IDs are not missed.
- Reranking: retrieve a few dozen candidates quickly, then reorder them with a slower, more precise model.
- Good chunking: chunks that are too large blur the meaning; too small and they lose context. Overlap helps.
- Measure it: build a set of real queries with known good answers and track recall at k.
Common mistakes
- Using different embedding models for documents and queries.
- Relying on vectors alone for queries full of identifiers or rare terms.
- Never re-embedding after changing the model or the chunking.
What interviewers ask about vector search
- "Cosine similarity or dot product?" They rank the same when vectors are normalised; cosine ignores length.
- "Why approximate search?" Exact search grows linearly with the data; ANN keeps latency low at millions of vectors.
- "Your RAG answers miss obvious documents. What do you check?" Chunking, hybrid search, the number of results retrieved and reranking.
What should you learn before Vector search and retrieval?
- ChunkingSplitting documents into passages sized for embedding and retrieval.
- EmbeddingsLists of numbers that place similar meanings close together, used for search and clustering.
What to learn after Vector search and retrieval
- Retrieval-augmented generation (RAG)Fetching relevant passages from your documents and giving them to the model with the question.
Read and watch: Vector search and retrieval explained
Practise Vector search and retrieval
- Intermediate Generative AI Interview Questions4 questions on Vector search and retrieval
- Generative AI Interview Questions for Freshers1 question on Vector search and retrieval
- Advanced Generative AI Interview Questions1 question on Vector search and retrieval
- Top-K Similar DocumentsCoding problem · medium