learn deep. ship safe.Practice testsFollow ByteMunSubscribe
Machine Learning · Senior

Advanced Machine Learning Interview Questions

12 machine learning interview questions for senior engineers and data scientists: batch normalization, vanishing gradients, dropout, drift, calibration, offline versus online results, train-serving skew, validation design, softmax gradients and PR-AUC. Each answer explains why the other options are wrong.

CoversDeep learningDriftImbalanced dataMetricsMLOpsModel evaluationOptimisationRegularization

Practice questions with answers

Tap an option: green is right, red is wrong, and the explanation says why.

0/10 answered · ✓ 0 ✗ 0
Question 1 of 101 / 10
Q1Choose one answermedium

What does batch normalization do in a neural network?

Show answer and explanation

Answer: C. It normalises a layer's inputs using mini-batch statistics, then rescales them, which stabilises and speeds up training

Batch normalization standardises each feature of a layer's input using the mean and variance of the current mini-batch, then applies a learned scale and shift. This keeps activations in a well-behaved range, allows higher learning rates and makes training less sensitive to initialisation. At inference it uses running averages instead of batch statistics. It doesn't filter data, it isn't how batches are formed, and making outputs sum to 1 is the job of softmax.

Q2Choose one answermedium

What is the vanishing gradient problem?

Show answer and explanation

Answer: B. Gradients shrink as they flow back through many layers, so early layers learn very slowly

Backpropagation multiplies gradients layer by layer; with saturating activations such as sigmoid or tanh, those factors are often below 1, so the signal reaching early layers becomes tiny and those layers barely update. Remedies include ReLU-family activations, careful initialisation, normalization layers and residual connections, which give gradients a direct path. Gradients growing too large is the opposite problem, exploding gradients, usually handled with gradient clipping.

Q3Choose one answermedium

How does dropout help a neural network?

Show answer and explanation

Answer: B. It randomly zeroes some activations during training, so the network can't rely on any single unit, which reduces overfitting

During training each unit is switched off with probability p on every step, so the network learns redundant, more general features instead of co-adapted ones; it behaves a bit like training an ensemble of thinned networks. At inference dropout is turned off and all units are used (with activations scaled accordingly). It doesn't prune neurons permanently, doesn't remove training examples, and doesn't speed up inference.

Q4Choose one answerhard

A model's accuracy slowly falls months after launch, with no code changes. What is the most likely explanation and response?

Show answer and explanation

Answer: B. Data or concept drift: the inputs or their relationship to the target changed, so monitor distributions and performance and retrain

Deployed weights don't change, but the world does: new customer segments, seasonal behaviour or fraudsters adapting all shift the input distribution (data drift) or the input-to-label relationship (concept drift). Production ML needs monitoring of feature distributions, prediction distributions and, when labels arrive, live performance, plus a retraining process triggered by schedule or by drift alerts. Hardware speed affects latency, not accuracy, and drift is manageable.

Q5Choose one answerhard

What does it mean for a classifier's probabilities to be well calibrated?

Show answer and explanation

Answer: D. Among predictions of 0.8, about 80% are actually positive; predicted probabilities match observed frequencies

Calibration is about whether probabilities can be taken at face value, which matters when they feed decisions such as pricing or risk thresholds. You check it with a reliability diagram or the Brier score, and fix it with Platt scaling or isotonic regression on a held-out set. Ranking quality (ROC-AUC) is separate: a model can rank perfectly yet be overconfident. Squeezed probabilities or epoch count don't define calibration.

Q6Choose one answerhard

A new model beats the old one on every offline metric, but the A/B test shows no improvement. What is the most likely reason?

Show answer and explanation

Answer: A. The offline metric doesn't capture the business outcome, or offline features differ from what the model sees in production

Offline evaluation optimises a proxy, say AUC on historical clicks, which may not move the online goal, such as revenue or retention. The gap also appears when offline data differs from live traffic: train-serving skew, feedback loops, or a test set from a different period. Check that the offline metric correlates with the online one, compare live feature distributions with training data, and make sure the A/B test had enough power. A/B tests do detect real improvements, and offline metrics are useful when they track the goal.

Q7Select all that applyhard

Which of these cause train-serving skew? Select all that apply.

Show answer and explanation

Answer: A. Different preprocessing code for training and for serving + B. Features computed with different freshness, such as daily batch values in training and real-time values in serving

Train-serving skew means the model sees inputs in production that differ from what it trained on. Two common sources are reimplementing feature logic separately (a Python notebook for training, Java in the service) and mismatched freshness or time windows. Sharing one pipeline or feature store, and training on logged serving features, are the standard fixes, not causes. Skew is worth testing for directly by comparing feature distributions between training data and live requests.

Q8Choose one answerhard

Why should hyperparameters be tuned on a validation set rather than the test set?

Show answer and explanation

Answer: C. Choosing settings by test-set score leaks the test set into model selection, so the final test score becomes optimistic

Every time you pick the hyperparameters that score best on a dataset, you fit that dataset a little. If that dataset is the test set, the reported test score no longer estimates performance on new data. The usual setup is train for fitting, validation (or cross-validation) for tuning and model selection, and a test set used once at the end. Validation sets don't raise accuracy, and nothing technical stops misuse, which is why it is a classic interview question.

Q9Choose one answerhard

For softmax followed by cross-entropy loss, what is the gradient of the loss with respect to the logits?

Show answer and explanation

Answer: C. The predicted probabilities minus the one-hot true label

Combining softmax with cross-entropy gives a remarkably simple gradient: for each class, pi − yi. If the true class has probability 0.7, its gradient is −0.3 (push its logit up) and every other class's gradient is its own probability (push down). That simplicity, plus numerical stability when computed together (log-softmax), is why frameworks fuse them. It isn't the raw logits, it is only zero when predictions are perfect, and the last option has the wrong sign and the wrong terms.

Q10Choose one answerhard

Positives are 1 in 1,000 examples. Why is PR-AUC often more informative than ROC-AUC here?

Show answer and explanation

Answer: C. ROC's false-positive rate is diluted by the huge number of negatives, while precision shows how many flagged cases are real

With 999 negatives per positive, a model can flag thousands of negatives while its false-positive rate stays tiny, so the ROC curve and its AUC still look excellent. Precision divides by everything flagged, so those false alarms show up clearly, and PR-AUC reflects what an analyst reviewing alerts actually experiences. ROC-AUC can be computed, PR-AUC is usually lower on rare-event problems, not higher, and it focuses on the positive class rather than ignoring it.

Take it as a timed test

  • 12 questions
  • 18 min
  • Pass mark 60%
  • Answers shown at the end

Sign in with your email to take the test: we send a 6-digit code, no password. Your scores and reports are saved to your account. The test also uses 10 questions that aren't shown above, so practise for understanding, not memory.

About this paper

This advanced paper has 12 machine learning questions for senior engineers and data scientists who train deep models and run them in production. The first part covers training: batch normalization, vanishing gradients, dropout, and the gradient of softmax with cross-entropy. The second covers what goes wrong after launch: data and concept drift, poorly calibrated probabilities, train-serving skew, offline gains that don't show up in an A/B test, and choosing metrics for very rare events. Ten questions are on this page with explanations. Each timed test draws 12 questions from a larger bank, including ones that aren't shown here, so every attempt is different.

In a real interview expect follow-ups on how you would detect and fix each problem, so read the explanations for the remedies as well as the definitions.

Written by Published Answers checked by Deepak

More Machine Learning quizzes

All Machine Learning quizzes →