Senior Analytics & AI Engineer: Two Numbers, One Partner Meeting

The Challenge

Measure Well helps cradle-to-career and place-based partnerships use data and AI to see what is working and improve outcomes for children and families. You are Measure Well's Senior Analytics & AI Engineer, the first technical teammate reporting to the Chief AI & Data Officer (CAIDO).

It is Monday afternoon. Tomorrow at 9:00 a.m. you meet with the CAIDO and the executive director of a partner organization. Three things are on your desk.

1. Two numbers that should match, don't. The CEO's weekly dashboard reports that 64% of partner organizations are "On Track" in their annual assessment. A program manager pulled the same metric from the warehouse and got 33%. The CEO will quote this number to funders on Friday.

2. The CEO also wants to know whether the Field Coach is helping. The Field Coach is Measure Well's Claude-based assistant that helps partner organizations understand their own data. The table below includes 30 days of Field Coach usage for each partner.

Here is the table both numbers came from (fct_partner_assessment, loaded nightly; data as of Monday 6:00 a.m.). It is illustrative data with the kinds of problems real partner data has.

partner_id	partner_name	consent_to_share	assessment_stage	region	coach_sessions_30d	coach_users_30d
P-001	Northside Promise	true	On Track	Midwest	42	6
P-002	Riverbend Cradle2Career	true	on_track	South	8	1
P-003	Eastgate Learning Partnership	true	Needs Support	Northeast	31	5
P-004	Harbor City Kids Collective	true	ON TRACK	Northeast	0	0
P-005	Mesa Futures	false	On Track	West	19	3
P-006	Prairie Pathways	true	Needs Support	Midwest	3	1
P-007	Lakeshore Youth Alliance	true	On Track	Midwest	27	4
P-008	Delta Youth Network	true	Not Started	South	0	0
P-009	Cedar Valley Partners	true	on track	Midwest	12	2
P-010	ZZ Test Org (do not use)	true	On Track	(blank)	64	1
P-011	Bayview Family Partnership	true	Needs Support	West	22	4
P-012	Piedmont Rising	true	Needs Support	South	5	1

3. Two Field Coach answers need your review. A program staff member reviews every Field Coach answer before a partner sees it. Before the tool goes to more partners, the CAIDO wants your view on these two:

• Riverbend Cradle2Career asked: "What share of our cohort is on track?"

  Field Coach answered: "Riverbend Cradle2Career is On Track. Across the network, 64% of partners are On Track, and peers like Lakeshore Youth Alliance and Mesa Futures are also On Track, a great sign for the Midwest and West."

• Eastgate Learning Partnership asked: "Why are we marked Needs Support?"

  Field Coach answered: "Your assessment showed missing attendance data for Q3. Partners who improve their attendance tracking typically move to On Track within six weeks."

Tomorrow, both people in the room will ask questions. They will want to know which number is right, what the dashboard will show going forward, what the usage data says, and whether the Field Coach is safe to put in front of more partners.

Constraints

• Define each metric once, in the model. Metrics are defined in reviewed, tested SQL (dbt), not in the dashboard tool. Your corrected metric must come from a SQL query with at least one test, and the dashboard reads from it.
• You don't own ingestion. An implementation partner runs the nightly loads from HubSpot and Airtable. You can't change the source systems or the loaders; you can model around problems and hand the partner a clear ticket.
• Consent and isolation come first. Every row must pass the consent check before it reaches anything partner-facing, and each partner organization may see only its own rows. Staff dashboards and partner-facing views may need different rules, say which is which.
• A human stays in the loop. The Field Coach never answers a partner without a program staff member reviewing it first. Your fixes should make that reviewer's job easier, not remove them.
• One drives, one approves. You drive these decisions; the CAIDO approves them. Be clear about what you decided, what you need approved, and what you'd tell the CEO before Friday.

On Using AI

We expect you to use AI tools. We evaluate how you use them, not whether you use them. Evidence of iteration, redirection, and critical evaluation scores higher than a polished output with no process documentation.

The single highest-signal indicator: your video answer to the mandatory AI question. If you cannot name a specific moment where you redirected AI output, evaluators will assume you did not.

Mandatory AI question for your video: Walk me through one moment where you disagreed with, pushed back on, or redirected what the AI gave you, and what you did instead. Name the specific moment. Explain what the AI produced that didn't meet the bar, what you did differently, and why.

Communication clarity matters for this role. We assess structure, audience awareness, and your ability to explain data to people who are not specialists, not presentation style or accent.

Tools: Use any dashboard or charting tool you can access for free: Tableau Public, Power BI Desktop, Looker Studio, an Omni trial, Excel or Google Sheets, or a chart built in Python or with an AI assistant. Tool choice is not scored. You don't need a database; write standard SQL and check it however you like. We care about how clearly your dashboard tells the story, not about tool features.

Submission

Upload each deliverable as a separate file directly on the Provn platform: your primary artifact, your README document (Sections A, B, and C), and your video walkthrough (MP4 or MOV).