learn deep. ship safe.Practice testsFollow ByteMunSubscribe
Generative AI · Senior

Advanced Generative AI Interview Questions

12 generative AI interview questions for senior engineers: KV cache, quantization, self-attention cost, positional encoding, speculative decoding, serving cost, reward models, agent loops and LLM-as-judge evaluation. Each answer explains why the other options are wrong.

CoversAgentsAlignmentCostEvaluationFine-tuningInferenceQuantizationTransformers

Practice questions with answers

Tap an option: green is right, red is wrong, and the explanation says why.

0/10 answered · ✓ 0 ✗ 0
Question 1 of 101 / 10
Q1Choose one answermedium

What does the KV cache do during LLM inference?

Show answer and explanation

Answer: D. Stores the attention keys and values of earlier tokens so each new token doesn't recompute them, trading memory for speed

Each generated token attends to all previous tokens, which needs their key and value vectors from every layer. Without a cache those would be recomputed at every step; the KV cache keeps them, so each step only computes the new token. The cost is GPU memory that grows with sequence length and batch size, which is often what limits how many requests a server handles. It is not a response cache, a session store or a vector index.

Q2Choose one answermedium

What does quantizing an LLM to 8-bit or 4-bit weights achieve?

Show answer and explanation

Answer: D. It stores weights with fewer bits, cutting memory and often speeding inference, at the risk of some quality loss

Weights are usually trained in 16- or 32-bit floating point. Quantization maps them to lower-precision formats such as 8-bit or 4-bit integers, so a model needs a half to a quarter of the memory and can fit on smaller GPUs, often running faster because memory bandwidth is the bottleneck. Good methods keep quality close to the original, but some loss is possible, so you measure it on your task. Precision goes down, not up, and vocabulary and placement are unrelated.

Q3Choose one answermedium

What does self-attention compute for each token in a transformer?

Show answer and explanation

Answer: D. A weighted mix of the value vectors of the tokens it can see, with weights from query and key similarity

Each token is projected into a query, a key and a value. Its query is compared with every visible token's key (a scaled dot product), a softmax turns those scores into weights, and the output is the weighted sum of the value vectors. That lets any token pull information from any other, near or far, in one step. It is not limited to the previous token, it is not a fixed-window convolution, and dropping tokens randomly is dropout, a training trick.

Q4Choose one answerhard

Why does the cost of standard self-attention grow quadratically with sequence length?

Show answer and explanation

Answer: C. Every token attends to every other token, so the score matrix has n × n entries

For a sequence of n tokens, attention computes a score for each pair of positions: an n by n matrix per head and layer. Doubling the context roughly quadruples that work and memory, which is why long contexts are expensive and why techniques such as FlashAttention (better memory use), sliding-window attention and KV caching matter. Vocabulary size affects the output layer, layer count adds cost linearly, and embedding size doesn't grow with the sequence.

Q5Choose one answerhard

Why do transformers need positional information such as positional encodings or rotary embeddings?

Show answer and explanation

Answer: A. Attention by itself ignores word order, so position signals tell the model where each token is

Self-attention treats its input as a set: shuffle the tokens and, without extra information, each token gets the same result. But "dog bites man" and "man bites dog" differ. Positional encodings add order: originally fixed sinusoidal vectors added to embeddings, and in many current LLMs rotary position embeddings (RoPE) that rotate queries and keys by position. They aren't document metadata, don't speed up hardware, and text models need them as much as vision models do.

Q6Choose one answerhard

How does speculative decoding speed up generation?

Show answer and explanation

Answer: D. A small draft model proposes several tokens and the large model checks them in one pass, keeping the ones it agrees with

Generating one token at a time with a large model is slow because each step is a full forward pass. In speculative decoding a cheap draft model guesses the next few tokens, then the large model scores all of them in a single parallel pass and accepts the longest prefix consistent with its own distribution, correcting the first mismatch. With the standard acceptance rule the output distribution matches the large model's, but latency drops when drafts are often right. It isn't prefetching user questions, batching unrelated prompts, or a temperature change.

Q7Select all that applyhard

Which of these reduce the cost or user-perceived latency of a production LLM feature? Select all that apply.

Show answer and explanation

Answer: B. Route simple requests to a smaller, cheaper model + C. Cache responses or reuse a cached prompt prefix for repeated requests + E. Stream tokens to the user as they are generated

Caching avoids paying again for identical work, and prompt-prefix caching lets providers skip reprocessing a long shared system prompt. Routing sends easy requests to a small model and keeps the large one for hard cases. Streaming doesn't make generation finish sooner, but users see words within a fraction of a second, which is what latency feels like. Stuffing every document into each prompt multiplies input cost, and a needlessly high max-tokens limit risks long, costly answers without helping quality.

Q8Choose one answerhard

In preference tuning such as RLHF, what is the reward model?

Show answer and explanation

Answer: C. A model trained on human comparisons of responses that scores how good a response is, used to optimise the LLM

In RLHF, people compare pairs of model answers and pick the better one. A reward model is trained on those comparisons to predict a score for any response, and the LLM is then optimised (for example with PPO) to produce answers the reward model scores highly, with a penalty for drifting too far from the original model. Methods such as DPO skip the separate reward model and learn from the comparisons directly. It isn't payroll, the pre-training loss, or a keyword filter.

Q9Choose one answerhard

An agent keeps calling the same tool over and over and never finishes. What is the best fix?

Show answer and explanation

Answer: B. Cap steps and budget, detect repeated calls, and return tool errors in a form the model can act on

Loops usually come from a tool result the model can't use (an unhelpful error, an empty result) or from missing stop conditions. Production agents need hard limits on steps, tokens and time; detection of repeated identical calls; and tool errors that explain what went wrong and what to try. A higher temperature makes behaviour less predictable without fixing the cause, removing the system prompt removes guidance, and more tools add more ways to get lost.

Q10Choose one answerhard

What is a known weakness of using an LLM as a judge to grade answers?

Show answer and explanation

Answer: A. Judges can favour longer answers, their own style or the answer shown first, so they need checking against human labels

LLM-as-judge scales evaluation of open-ended output, but judges have measurable biases: preferring verbose answers, answers in their own style, or whichever option appears first in a pairwise comparison. Good practice is to write a clear rubric, randomise answer order, compare the judge's scores with a set of human labels to measure agreement, and re-check when you change the judge model. It is not perfectly aligned with humans, not limited to maths, and widely used.

Take it as a timed test

  • 12 questions
  • 18 min
  • Pass mark 60%
  • Answers shown at the end

Sign in with your email to take the test: we send a 6-digit code, no password. Your scores and reports are saved to your account. The test also uses 10 questions that aren't shown above, so practise for understanding, not memory.

About this paper

This advanced paper has 12 generative AI questions for senior and lead engineers who own LLM systems in production. It goes under the API: why attention is quadratic, what the KV cache trades, how quantization and speculative decoding cut cost and latency, and what a reward model does in preference tuning. It then covers the problems that appear at scale: agents stuck in loops, serving budgets, and the biases of using an LLM to grade another LLM. Ten questions are on this page with explanations. Each timed test draws 12 questions from a larger bank, including ones that aren't shown here, so every attempt is different.

Expect follow-ups in a real interview: be ready to explain the trade-off behind each answer, not only the definition.

Written by Published Answers checked by Deepak

More Generative AI quizzes

All Generative AI quizzes →