Grade the Agent’s Answer

Marlowe Athletic has been piloting an internal analytics assistant called Ask Marlowe. Club general managers type a question in plain English; the assistant writes SQL, runs it, and returns a table plus a recommendation. GMs are not…

National AI Competition
Wave 2
Data Analyst / Data Scientist Career Ladder
Estimated Time:
40 minutes
Difficulty:Intermediate
Status:Not started
Start this challenge

Create a free account to upload your work. Your progress saves as a draft until you submit.

What You'll Be Doing

Marlowe Athletic has been piloting an internal analytics assistant called Ask Marlowe. Club general managers type a question in plain English; the assistant writes SQL, runs it, and returns a table plus a recommendation. GMs are not analysts.

Eight weeks ago the GM of Club 41 asked: “Which of my class formats bring in members who stay?”

Here is the schema Ask Marlowe has access to.

members(member_id, club_id, tier, joined_at, cancelled_at)
class_bookings(booking_id, member_id, class_id, booked_at, attended)
classes(class_id, club_id, format, start_at, capacity)

Here is the query it wrote.

SELECT c.format, COUNT(DISTINCT b.member_id) AS bookers, ROUND(100.0 * COUNT(DISTINCT CASE WHEN m.cancelled_at IS NULL THEN b.member_id END) / COUNT(DISTINCT b.member_id), 1) AS pct_still_active FROM class_bookings b JOIN classes c ON b.class_id = c.class_id JOIN members m ON m.member_id = b.member_id WHERE [c.club](https://c.club)

_id = 41 AND b.booked_at >= '2025-09-01' GROUP BY c.format ORDER BY pct_still_active DESC;

Here is what it returned.

And here is the recommendation it gave the GM:

FormatBookerspct_still_active
Aqua Fit41288.1
Cycle 451,93381.4
Yoga Flow1,61079.8
Strength Lab1,20476.2
Anvil68858.3
Sunrise HIIT50155.9

“Aqua Fit and Cycle 45 drive the strongest member retention at your club. Anvil and Sunrise HIIT show the weakest retention among their attendees and are candidates for removal from the schedule.”

The GM cut Anvil and Sunrise HIIT from the Club 41 schedule six weeks ago. The regional director has now asked the data team to check the work.

You pull one more thing before you start — the member mix behind each format.

FormatBookersMedian months since joining, at time of booking% of bookings made in a member’s first 90 days
Aqua Fit412516%
Cycle 451,9333411%
Yoga Flow1,6102914%
Strength Lab1,2042219%
Anvil688561%
Sunrise HIIT501466%

And one club-wide baseline, which the data team already publishes: at Club 41, members in their first 90 days have a 12-month survival rate of 57%. Members past 24 months have a 12-month survival rate of 87%.

Write the regional director a memo.

Constraints to consider

  • You have read-only access to the three tables shown. No new tables, no new columns, no additional event tracking, no survey.
  • You cannot answer “we need more data” and stop. The regional director needs a position on the two cut formats, using what is in front of you.
  • The two formats were cut six weeks ago. Whatever you recommend has to account for six weeks of members who have already lost them.
  • Ask Marlowe stays on. It will keep answering GM questions across all 84 clubs regardless of what you find. Any fix you propose has to be a change to how the assistant behaves, not a request to switch it off.
  • Thirty minutes. One corrected number is worth more to the regional director than five directional statements.

AI Usage Guidance

We expect you to use AI tools. We evaluate how you use them — not whether you use them. Evidence of iteration, redirection, and critical evaluation scores higher than a polished output with no process documentation.

The single highest-signal indicator: your video answer to the mandatory AI question. If you cannot name a specific moment where you redirected AI output, evaluators will assume you did not.

Mandatory AI question for your video: Walk me through one moment where you disagreed with, pushed back on, or redirected what the AI gave you — and what you did instead. Name the specific moment. Explain what the AI produced that didn’t meet the bar, what you did differently, and why.

Speak naturally. Communication is assessed on clarity of technical ideas and logical structure — not verbal polish, accent, or filler words.

What You'll Accomplish

Distinguish a query that is incorrect from a query that is correct and answers the wrong question

Identify a confound in observational data and adjust a comparison for it

Produce a corrected recommendation from the same data that produced a wrong one

Design a behavioral change to an AI analytics tool that prevents a class of error, not one instance of it

Communicate a reversal to a stakeholder whose team has already acted on the wrong answer

How Your Work Will Be Scored

Analytical Rigor & Query Reasoning Evaluation of AI-Generated Analysis Insight-to-Recommendation Video Walkthrough & Communication AI Fluency

What to Submit

File 1 — Memo to the Regional Director

Document · 2 pages maximumRequired

Format: .pdf, .doc, .docx, .rtf, .txt, .md

What the assistant’s answer actually measured, what the corrected answer is, and what should happen to Anvil and Sunrise HIIT at Club 41.

Sign in to upload files

File 2 — README Document

DocumentRequired

Format: .pdf, .doc, .docx, .rtf, .txt, .md

Three required sections: 

  • Section A — Written analysis: the technical account. What the query computes, why that is not what the GM asked for, and how the member-mix table changes the ranking. Show at least one corrected or adjusted comparison with the arithmetic visible. 350–450 words. 
  • Section B — The fix to Ask Marlowe: (i) the change you would make to how the assistant answers questions of this shape — stated as a mechanism, not a warning label; (ii) one check that would have caught this specific answer before it reached the GM; (iii) whether that check is automated or needs a human, and why. 
  • Section C — AI Usage Log (Mandatory): This is not a trick. We want to see how you work with AI — not whether you used it. In a short section of your README, document your AI collaboration process. For each significant interaction with an AI tool, briefly note: what you asked the AI to help with / what it gave you / what you kept, changed, or rejected — and why. Three interactions documented is sufficient. The log does not need to be exhaustive.

Sign in to upload files

File 3 — Video Walkthrough

Video · 5-6 minutesRequired

Format: .mp4, .mov, .webm

Record as MP4 or MOV and upload directly on the Provn platform as a separate file. Cover: (1) what the assistant’s answer actually measured, in one sentence, ~30 sec; (2) the corrected comparison and how you got there, ~90 sec; (3) your fix to Ask Marlowe and the check behind it, ~90 sec; (4) the mandatory AI question, ~60 sec; (5) what you are least sure about, ~45 sec. Speak naturally — we assess reasoning, not verbal polish.

Sign in to upload files