Intermediate Machine Learning Interview Questions
12 machine learning interview questions for practitioners with some experience: L1 and L2 regularization, data leakage, random forests, gradient boosting, ROC-AUC, imbalanced classes, standardisation, learning rates and thresholds. Each answer explains why the other options are wrong.
CoversData leakageEnsemblesFeature engineeringImbalanced dataMetricsModel evaluationOptimisationRegularization
Practice questions with answers
Tap an option: green is right, red is wrong, and the explanation says why.
What does L2 regularization (ridge) add to a model's training objective?
Show answer and explanation
Answer: A. A penalty on the sum of squared weights, which discourages large weights and reduces overfitting
L2 regularization adds λ times the sum of squared weights to the loss, so the optimiser trades a little training fit for smaller, smoother weights that generalise better; λ controls the strength. It doesn't add data, change the architecture or delete features. In neural networks the same idea appears as weight decay, and interviewers often follow up by asking how L1 differs.
Which of these is an example of data leakage?
Show answer and explanation
Answer: B. Fitting a scaler on the full dataset before splitting it into train and test sets
Leakage is when information that wouldn't be available at prediction time reaches training. Fitting a scaler (or an imputer or feature selector) on all the data lets test-set statistics shape the training features, so test scores come out optimistic. Fit preprocessing on the training split only, ideally inside a pipeline. Larger training sets, shuffling (for non-time-series data) and more epochs are normal practice, not leakage.
What is a key practical difference between L1 and L2 regularization?
Show answer and explanation
Answer: C. L1 can push some weights to exactly zero, effectively selecting features; L2 shrinks weights but rarely to zero
The L1 penalty (sum of absolute weights) has a corner at zero, so the optimum often sets some weights exactly to zero, giving a sparse model that ignores those features; this is lasso. L2 (sum of squares) pulls weights smoothly toward zero without zeroing them. Both work for linear models and neural networks, and elastic net combines them when you want sparsity plus stability with correlated features.
Why does a random forest usually generalise better than a single deep decision tree?
Show answer and explanation
Answer: B. It averages many trees trained on bootstrap samples with random feature subsets, which reduces variance
A single deep tree has low bias but high variance: small changes in data give a different tree. A random forest trains many trees, each on a bootstrap sample and considering a random subset of features at each split, so their errors are less correlated, and averaging them cancels much of the variance. Forest trees are usually deep, not shallow. Training trees sequentially to fix errors describes boosting, and the combination is a simple vote or average.
How does gradient boosting build its model?
Show answer and explanation
Answer: B. It adds trees one at a time, each fitted to the remaining errors (the loss gradient) of the ensemble so far
Boosting is sequential: start with a simple prediction, then repeatedly fit a small tree to the negative gradient of the loss (for squared error, the residuals) and add it with a small learning rate. Each tree corrects what the ensemble still gets wrong, which is why boosted trees (XGBoost, LightGBM, CatBoost) are so strong on tabular data, and why they can overfit if you add too many trees. Independent parallel trees describe bagging and random forests.
What does a ROC-AUC of 0.85 mean?
Show answer and explanation
Answer: C. If you pick a random positive and a random negative, the model ranks the positive higher 85% of the time
ROC-AUC summarises the ROC curve across all thresholds, and it has a neat interpretation: the probability that a randomly chosen positive gets a higher score than a randomly chosen negative. 0.5 is random ranking and 1.0 is perfect. It isn't accuracy, it isn't recall at one threshold, and it says nothing about calibrated confidence; a model can rank well while its probabilities are badly calibrated.
Which of these are sensible ways to handle a heavily imbalanced classification problem? Select all that apply.
Show answer and explanation
Answer: A. Use class weights so mistakes on the rare class cost more + C. Judge models with precision, recall, F1 or PR-AUC instead of accuracy + D. Tune the decision threshold for the business cost of each kind of error + E. Resample: oversample the minority class or undersample the majority class in the training data
Class weights and resampling make the model pay attention to the rare class (resample only the training data, never the test set). Metrics such as recall, precision, F1 and PR-AUC show performance on that class, which accuracy hides. Because the default 0.5 threshold is arbitrary, choosing the threshold from the cost of false positives versus false negatives often helps most. Deleting the minority class removes the very thing you are trying to predict.
What does this code print?
xs = [2, 4, 6]
mean = sum(xs) / len(xs)
std = (sum((x - mean) ** 2 for x in xs) / len(xs)) ** 0.5
print([round((x - mean) / std, 2) for x in xs])Show answer and explanation
Answer: A. [-1.22, 0.0, 1.22]
This is standardisation (z-scores). The mean is 4. The population variance is ((−2)² + 0² + 2²) / 3 = 8/3, so the standard deviation is about 1.633. Dividing −2, 0 and 2 by 1.633 gives −1.22, 0.0 and 1.22. [-1.0, 0.0, 1.0] would need a standard deviation of 2, [-2.0, 0.0, 2.0] skips the division, and [0.0, 0.5, 1.0] is min-max scaling, a different technique.
During gradient descent the training loss jumps around and then grows. What is the most likely cause?
Show answer and explanation
Answer: C. The learning rate is too high, so updates overshoot the minimum
Each step moves the weights by the learning rate times the gradient. If the rate is too large, steps overshoot the valley floor, the loss oscillates and can diverge, sometimes to NaN. Lower the learning rate, use a schedule or warm-up, or an adaptive optimiser such as Adam, and check input scaling. A rate that is too low gives slow, steady progress rather than divergence, and model size or dataset size don't cause this pattern.
A fraud model flags transactions above a score threshold. What happens when you lower the threshold?
Show answer and explanation
Answer: A. Recall goes up because more fraud is caught, but precision usually goes down because more legitimate transactions are flagged
A lower threshold labels more transactions as fraud, so fewer real frauds slip through (higher recall) but more good customers get flagged (more false positives, lower precision). Raising it does the reverse. The model's scores don't change, only the decision rule, which is why the threshold should be chosen from the business cost of each error, often by reading the precision-recall curve. Both rising together would need a better model, not a different threshold.
Take it as a timed test
- 12 questions
- 15 min
- Pass mark 60%
- Answers shown at the end
Sign in with your email to take the test: we send a 6-digit code, no password. Your scores and reports are saved to your account. The test also uses 10 questions that aren't shown above, so practise for understanding, not memory.
About this paper
This intermediate paper has 12 machine learning questions for people who have trained models on real data. It covers the choices that separate a working model from a misleading one: L1 versus L2 regularization, how leakage sneaks into features and preprocessing, why random forests and gradient boosting behave differently, what ROC-AUC really measures, and how to handle imbalanced classes and decision thresholds. One question asks you to predict the output of a short standardisation snippet. Ten questions are on this page with explanations. Each timed test draws 12 questions from a larger bank, including ones that aren't shown here, so every attempt is different.
Interviewers at this level often ask "how would you check that?", so read the explanations for the reasoning, not only the answer.