The Agent That Can Spend Money

Halcyon wants Deskmate to stop drafting and start acting. The proposal: Deskmate tenders loads to carriers directly and issues rate confirmations — the document that binds Halcyon’s broker customer to a price. Money moves. It has been…

National AI Competition
Wave 3
AI / Agentic Software Engineer Career Ladder
Estimated Time:
40 minutes
Difficulty:Intermediate
Status:Not started
Start this challenge

Create a free account to upload your work. Your progress saves as a draft until you submit.

What You'll Be Doing

Halcyon wants Deskmate to stop drafting and start acting. The proposal: Deskmate tenders loads to carriers directly and issues rate confirmations — the document that binds Halcyon’s broker customer to a price. Money moves. 

It has been running in shadow mode internally for six weeks and in early access with six brokerages for three. You own the go/no-go. 

Here is what you know: 

  • On a 240-case internal suite, the agent completes the full tender flow correctly in 86% of cases. Of the 14% that fail: 8% are wrong-rate tenders (a binding rate confirmation at the wrong price) and 6% are incomplete tenders (the carrier never gets a usable document). 
  • Early access CSAT is 4.5, from six brokerages who volunteered. 
  • p50 latency 6s; p95 41s. 
  • Cost per processed email is $0.09. At the projected 41,000 emails/month that is $3,690/month against a category finance budgeted at $1,500/month. 
  • Legal has not signed off on the agent issuing a rate confirmation without a human approval step. 
  • Two of the three engineers roll off to the carrier-portal migration in five weeks, whatever you decide. 

Here is the tool definition and the handler, as currently written. 

# halcyon/deskmate/tools/tender.py 
TENDER_SCHEMA = { 
    "name": "tender_load", 
    "description": "Tender a load to a carrier and issue a rate confirmation.", 
    "parameters": { 
        "type": "object", 
        "properties": { 
            "broker_id":   {"type": "string"}, 
            "load_id":     {"type": "string"}, 
            "carrier_scac":{"type": "string"}, 
            "rate_cents":  {"type": "integer"}, 
            "max_rate_cents": {"type": "integer"},   # optional 
        }, 
        "required": ["broker_id", "load_id", "carrier_scac", "rate_cents"], 
    }, 
} 
 
def tender_load(broker_id, load_id, carrier_scac, rate_cents, 
                max_rate_cents=None): 
    if not user_can_tender(broker_id): 
        raise PermissionError("not authorized") 
    if max_rate_cents and rate_cents > max_rate_cents: 
        raise ValueError("rate above ceiling") 
    return tms.post("/tenders", { 
        "broker_id": broker_id, "load_id": load_id, 
        "carrier_scac": carrier_scac, "rate_cents": rate_cents, 
    }) 

Write the readiness bar and make the call. 

Constraints to consider 

  • You cannot retrain the model or change vendors. You can change the prompt, the scaffolding, the tools, the retrieval, the guardrails, and the interface. 
  • The $1,500/month budget is fixed for the quarter. Finance will not revisit it. 
  • Whatever ships must be operable by one engineer after week five. 
  • Any capability that issues a binding rate confirmation without human approval requires documented legal sign-off before launch. No launch-then-fix, no parallel-track sign-off. 
  • You will present to a leadership group that includes Halcyon’s VP of Operations, who thinks handing tendering authority to a language model is reckless. 

AI Usage Guidance 

We expect you to use AI tools. We evaluate how you use them — not whether you use them. Evidence of iteration, redirection, and critical evaluation scores higher than a polished output with no process documentation. 

The single highest-signal indicator: your video answer to the mandatory AI question. If you cannot name a specific moment where you redirected AI output, evaluators will assume you did not. 

Mandatory AI question for your video: Walk me through one moment where you disagreed with, pushed back on, or redirected what the AI gave you — and what you did instead. Name the specific moment. Explain what the AI produced that didn’t meet the bar, what you did differently, and why. 

Speak naturally. Communication is assessed on clarity of technical ideas and logical structure — not verbal polish, accent, or filler words.

What You'll Accomplish

Define launch readiness for an agent that takes consequential actions, with thresholds rather than adjectives

Audit an agent tool interface for authorization and idempotency defects

Design guardrails that are mechanisms in code rather than statements of intent

Reason explicitly about the capability / latency / cost tradeoff with real numbers against a fixed budget

Defend a go/no-go decision to a leadership audience that includes an informed skeptic

How Your Work Will Be Scored

Readiness & Eval Design Agent Authorization & Guardrails Engineering Judgment: Cost, Latency, RollbackVideo Walkthrough & Communication AI Fluency

What to Submit

File 1 — Launch Readiness Brief

Document · 2 pages maximumRequired

Format: .pdf, .doc, .docx, .rtf, .txt, .md

Your readiness criteria with explicit numeric thresholds, your go / limited-GA / no-go recommendation, and the five-week plan behind it.

Sign in to upload files

File 2 — README Document

DocumentRequired

Format: .pdf, .doc, .docx, .rtf, .txt, .md

Three required sections: 

  • Section A — Written analysis: the tradeoff argument, with the numbers attached. Include how you get from $3,690/month to $1,500/month, or why you would not launch at that volume. State what you are trading against what. 350–450 words. 
  • Section B — The tool audit, the guardrails, and the rollback. Three parts: 
    • (i) Tool audit. Review the tender_load schema and handler above. Name every defect you find and rewrite the parts that need rewriting — as code, not prose. Say which defect you consider most severe and why. 
    • (ii) Readiness and guardrails. The eval suite you would run before launch: what the task suite measures, roughly how big, how success is judged per case, and the threshold that gates the launch. Then: the one failure mode you consider unacceptable at any rate, and the guardrail that prevents it. The guardrail must be a mechanism. 
    • (iii) Rollback. The trigger (numeric), the owner (a role), and how long it takes to execute. Say what happens to tenders already issued when it fires. 
  • Section C — AI Usage Log (Mandatory): This is not a trick. We want to see how you work with AI — not whether you used it. In a short section of your README, document your AI collaboration process. For each significant interaction with an AI tool, briefly note: what you asked the AI to help with / what it gave you / what you kept, changed, or rejected — and why. Three interactions documented is sufficient. The log does not need to be exhaustive.

Sign in to upload files

File 3 — Video Walkthrough

Video · 6-8 minutesRequired

Format: .mp4, .mov, .webm

Record as MP4 or MOV and upload directly on the Provn platform as a separate file. Cover: (1) your recommendation and the readiness bar, ~60 sec; (2) the tool audit — walk the code and show what you changed, ~2 min; (3) the eval suite and the gating threshold, ~90 sec; (4) the cost and latency tradeoff with the numbers, ~60 sec; (5) how you answer the VP of Operations, ~60 sec; (6) the mandatory AI question, ~60 sec; plus ~30 sec for what you would do differently with more time. Speak naturally, as if briefing your leadership team directly — technical clarity matters, verbal polish does not.

Sign in to upload files