What does a machine learning engineer do?
A machine learning engineer turns a model into a working part of a product: getting training data, building the pipeline that trains and checks the model, serving its predictions fast enough, and watching it once real users depend on it. Most of the job is engineering around the model, with the model itself a smaller share.
The Bureau of Labor Statistics has no profile for machine learning engineers. The nearest is computer and information research scientists, who BLS says "design innovative uses for new and existing computing technology." One listed duty is to design and conduct experiments to test software systems, "frequently using techniques from data science and machine learning." BLS gives a median annual wage of $140,300 in May 2025 and projects employment to grow 22 percent from 2025 to 2035.
O*NET OnLine has no machine learning engineer occupation either. The closest is data scientists (15-2051.00), whose description includes applying "data mining, data modeling, natural language processing, and machine learning" to large datasets. Its tasks include "Test, validate, and reformulate models to ensure accurate prediction of outcomes of interest."
How were these machine learning engineer questions chosen?
Each machine learning engineer question here comes from a published duty or from the practices in two Google engineering guides, matched to what an interviewer would test. The questions are drawn from the job's published duties, not from a survey of real interviews, and they do not describe any named company's loop.
| Duty or practice (BLS, O*NET, Google) | Question it tends to produce | What the interviewer is checking |
|---|---|---|
| Frame the problem and the metric | "Would you use a model here at all?" | A baseline first, and a metric tied to the product |
| Test, validate, and reformulate models | "How would you evaluate this model before launch?" | A split that matches real use, and errors you looked at |
| Compare models using performance metrics | "Model B scores higher offline. Do you ship it?" | Offline gains checked against an online test |
| Build training and serving pipelines | "Why does the model do worse in production?" | Training-serving skew and how to find it |
| Automate retraining and deployment | "How often should this model retrain?" | Triggers, validation gates and a way back |
| Monitor models after launch | "How would you know the model is getting worse?" | Input, output and outcome checks with alerts |
What problem-framing questions come up?
Framing questions test whether a machine learning engineer knows when a model is worth building. Google's Rules of Machine Learning open with "Rule #1: Don’t be afraid to launch a product without machine learning." Interviewers want a baseline, a metric that matches the product goal, and a plan for the first simple version.
1. We want to rank support tickets by urgency. How would you approach it?
What they listen for: questions about the data and labels first, a simple rule as a baseline, then a first model, and a metric agreed with the support team. Follow-up that tends to come next: "What would make you stop and keep the rule?"
2. How would you choose the metric for this model?
What they listen for: one metric you can observe and explain, tied to what the business cares about, plus guardrail metrics that must not get worse. Follow-up: "What if the metric goes up and users complain?"
3. Why start with a simple model?
What they listen for: the guide's Rule #4, "Keep the first model simple and get the infrastructure right," and the reason behind it. A simple model shows whether the pipeline and data work before you add complexity. Follow-up: "When would you move to a deep model?"
What evaluation questions should you expect?
Evaluation questions check whether a machine learning engineer can prove a model works on unseen data. The Rules of Machine Learning advise: "If you produce a model based on the data until January 5th, test the model on the data from January 6th and after." Interviewers want that split, plus error analysis.
4. How would you split data to evaluate a model that predicts next week's demand?
What they listen for: a time-based split, no future data leaking into training, and a check against a simple baseline such as last week's value. Follow-up: "How would you tune hyperparameters without touching the test set?"
5. Model B beats model A offline. Do you ship it?
What they listen for: an online test before full rollout, a look at where B is worse, and the cost of serving B. O*NET lists "Compare models using statistical performance metrics, such as loss functions or proportion of explained variance." Follow-up: "The online test shows no change. Now what?"
6. Your classes are 99 to 1. How do you evaluate the model?
What they listen for: why accuracy misleads here, precision and recall at a threshold the product needs, and looking at real examples of each error. Follow-up: "Who decides the threshold?"
What pipeline and production questions come up?
Production questions test whether a machine learning engineer can keep a model correct once it leaves the notebook. Google Cloud's guide to MLOps pipelines notes that "only a small fraction of a real-world ML system is composed of the ML code." Interviewers probe the data pipeline, serving and the gap between training and production.
7. The model did well offline but worse in production. Why?
What they listen for: training-serving skew and a method to find it. The Rules of Machine Learning define skew as "a difference between performance during training and performance during serving," caused by different data handling or by data changing between training and serving. Follow-up that tends to come next: "How would you stop it happening again?"
This question best shows the role, so here is a weak answer and the same answer fixed. Both are made-up examples.
Weak made-up example: "Production data is just different, so the model overfit. I'd retrain it on more data and add regularisation."
The weak answer guesses at a cause and jumps to retraining. It never checks whether the model sees the same features in production that it saw in training.
Fixed made-up example: "First I'd check for skew. I'd log the features the model actually receives at serving time for a sample of requests and compare them with the training features for the same users. In a case like this I'd look for a feature computed differently in the two pipelines, such as a 30-day count that serving fills in from a stale table. If I found that, I'd fix it so both pipelines share one feature code path, retrain, and add a daily check that compares serving and training feature distributions and alerts when they drift apart."
8. How would you serve predictions within 50 milliseconds?
What they listen for: precomputing what you can, a smaller or distilled model, caching, batching, and measuring latency at the tail. Follow-up: "What do you return if the model times out?"
9. What should be tested in an ML pipeline besides the model?
What they listen for: data schema checks, feature value checks and infrastructure tests. Rule #5 says "Test the infrastructure independently from the machine learning." Follow-up: "Which of those checks would block a deploy?"
What retraining and monitoring questions come up?
Monitoring questions check that a machine learning engineer can spot a model getting worse before users do. The MLOps guide says "In practice, models often break when they are deployed in the real world," and lists the lack of performance monitoring as a weakness of manual processes. Interviewers want inputs, outputs and outcomes watched, with a plan to act.
10. How would you know this model is getting worse?
What they listen for: alerts on input distributions, prediction distributions and the business outcome once labels arrive. Rule #10 warns: "Watch for silent failures." Follow-up: "Labels arrive 30 days late. What do you watch until then?"
11. How often should this model retrain?
What they listen for: retraining on a schedule or on a trigger such as data drift, with automatic validation before the new model serves traffic. The MLOps guide says the goal of its level 1 "is to perform continuous training of the model by automating the ML pipeline." Follow-up: "What stops a bad model from going live?"
12. A new model is live and complaints jump. What do you do?
What they listen for: roll back to the last good model, tell the team, then compare the two models on the complaint cases. Follow-up: "How fast can you roll back today?"
Which behavioral questions should you expect?
Behavioral questions ask a machine learning engineer about past projects, so answer each with one real event in STAR order: situation, task, action, result. Interviewers listen for your own part, the trade-offs you made, and what you learned when a model failed. The STAR method guide covers the structure.
13. Tell me about a model that did not work in production.
What they listen for: how you found out, what the cause was, and the check you added. Follow-up: "What would have caught it earlier?"
14. Tell me about a time you argued against using machine learning.
What they listen for: the evidence you brought, the simpler option you proposed, and how the decision was made. The page on using data to make a decision fits this question. Follow-up: "Were you right?"
What follow-up questions tend to come next?
Follow-ups test whether your first answer survives contact with real data. Expect the interviewer to remove a feature, delay the labels, double the traffic or halve the latency budget, then ask what changes. Prepare by asking yourself what breaks first in each system on your resume.
- "What happens if that feature stops arriving?"
- "How much data did you train on, and how was it labelled?"
- "What did the model get wrong most often?"
- "How much does each prediction cost to serve?"
- "Who uses the predictions, and what do they do with them?"
How should you prepare for a machine learning engineer interview?
Prepare two projects you can explain end to end: the problem, the data, the baseline, the model, how you evaluated it, how it was served and what happened after launch. Practise one ML system design out loud with a timer. Revise evaluation metrics and the reasoning behind each one.
The free interview question generator takes the posting and the round you pick, and returns likely questions, why they come up and what the interviewer listens for. Machine learning loops overlap with nearby roles, so compare this page with the questions for an AI engineer, a data scientist and a data engineer.
If you are moving into the field from another role, the guide to a career change into AI roles covers the resume side. Prept's Mock room lets you answer out loud on a clock, scores each answer out of 10 and checks it against your resume, and it is included in the free plan.