Machine learning theory and cases

Explain generalization, leakage, metrics, calibration and model interpretation with supplied evidence.

6 scenarios
35 minutes each
Speak or type your answers
A report at your level

A computer voice asks the questions.

Your interview level

JuniorBuild a clear answer and explain the fundamentals.MidWork independently and explain the trade-offs.SeniorHandle ambiguity, failure modes, and wider consequences.

Practice scenarios

Choose a scenario. Each interview ends with a report for the level you selected.

35 min4 stagesMid level

A demand model with a wide validation gap

At fictional Tern Retail, model A has training MAE 1 and validation MAE 9 units. Model B has training MAE 5 and validation MAE 6 units. Both use the same chronological split. These are supplied review figures, not measured employer results. Decide what the gap supports before choosing the next experiment.

Start interview

Rubric and preparationWhat a mid answer covers · 1 lesson to prepare

Mid standard

Choose a validation protocol that matches the deployment population and timing.
Connect the model comparison to a business cost, capacity constraint or explicit loss.
Defend a competing explanation and name evidence that would change the conclusion.

Lessons to prepare

35 min4 stagesMid level

A fraud metric under review capacity

Fictional Brookline Payments supplies two operating points over 100 mature labeled payments: A has TP=8, FP=10, FN=2, TN=80; B has TP=6, FP=4, FN=4, TN=86. Reviewers can inspect at most 12 payments from this population. All numbers are exercise inputs. Choose a feasible operating point and explain what evidence is still missing.

Start interview

Rubric and preparationWhat a mid answer covers · 1 lesson to prepare

Mid standard

Choose a validation protocol that matches the deployment population and timing.
Connect the model comparison to a business cost, capacity constraint or explicit loss.
Defend a competing explanation and name evidence that would change the conclusion.

Lessons to prepare

35 min4 stagesMid level

A churn model that looks too good

Fictional Juniper Subscriptions predicts cancellation at the start of each account-month. A random row split gives an almost perfect score. Repeated months of the same account occur in both partitions; refund_issued is filled after cancellation; the imputer was fitted before splitting. Investigate the result before it reaches a retention team.

Start interview

Rubric and preparationWhat a mid answer covers · 1 lesson to prepare

Mid standard

Choose a validation protocol that matches the deployment population and timing.
Connect the model comparison to a business cost, capacity constraint or explicit loss.
Defend a competing explanation and name evidence that would change the conclusion.

Lessons to prepare

35 min4 stagesMid level

Cross-validation for repeated clinic visits

Fictional Rowan Scheduling predicts a missed visit when the reminder list is created. A patient can have several visits. The launch population is next month at existing clinics. One clinic joins later. Choose validation that reflects those uses while protecting patient-level information.

Start interview

Rubric and preparationWhat a mid answer covers · 1 lesson to prepare

Mid standard

Choose a validation protocol that matches the deployment population and timing.
Connect the model comparison to a business cost, capacity constraint or explicit loss.
Defend a competing explanation and name evidence that would change the conclusion.

Lessons to prepare

35 min4 stagesMid level

A ranked risk score with unreliable probabilities

Fictional Cove Travel has a cancellation-risk model. In a supplied mature review bin, 20 bookings have predicted probability 0.8 and 8 cancel. The ranking still separates more cancellations near the top. Explain why ranking quality doesn't settle whether those probabilities can price a retention action.

Start interview

Rubric and preparationWhat a mid answer covers · 1 lesson to prepare

Mid standard

Choose a validation protocol that matches the deployment population and timing.
Connect the model comparison to a business cost, capacity constraint or explicit loss.
Defend a competing explanation and name evidence that would change the conclusion.

Lessons to prepare

35 min4 stagesMid level

A leading feature and an unsupported causal claim

Fictional Reed Logistics predicts late deliveries. Distance is highly ranked by permutation importance. Route type and distance are correlated. A manager concludes that shortening any route by one mile will reduce lateness by the model coefficient. Review the claim and propose evidence for an operational decision.

Start interview

Rubric and preparationWhat a mid answer covers · 1 lesson to prepare

Mid standard

Choose a validation protocol that matches the deployment population and timing.
Connect the model comparison to a business cost, capacity constraint or explicit loss.
Defend a competing explanation and name evidence that would change the conclusion.

Choose your character

Machine learning theory and cases

Practice scenarios

A demand model with a wide validation gap

Mid standard

Lessons to prepare

A fraud metric under review capacity

Mid standard

Lessons to prepare

A churn model that looks too good

Mid standard

Lessons to prepare

Cross-validation for repeated clinic visits

Mid standard

Lessons to prepare

A ranked risk score with unreliable probabilities

Mid standard

Lessons to prepare

A leading feature and an unsupported causal claim

Mid standard

Lessons to prepare

Interview Hub access

Your Work Gym account