The Agent That Can Spend Money
Halcyon wants Deskmate to stop drafting and start acting. The proposal: Deskmate tenders loads to carriers directly and issues rate confirmations — the document that binds Halcyon’s broker customer to a price. Money moves. It has been…
Create a free account to upload your work. Your progress saves as a draft until you submit.
What You'll Be Doing
Halcyon wants Deskmate to stop drafting and start acting. The proposal: Deskmate tenders loads to carriers directly and issues rate confirmations — the document that binds Halcyon’s broker customer to a price. Money moves.
It has been running in shadow mode internally for six weeks and in early access with six brokerages for three. You own the go/no-go.
Here is what you know:
- On a 240-case internal suite, the agent completes the full tender flow correctly in 86% of cases. Of the 14% that fail: 8% are wrong-rate tenders (a binding rate confirmation at the wrong price) and 6% are incomplete tenders (the carrier never gets a usable document).
- Early access CSAT is 4.5, from six brokerages who volunteered.
- p50 latency 6s; p95 41s.
- Cost per processed email is $0.09. At the projected 41,000 emails/month that is $3,690/month against a category finance budgeted at $1,500/month.
- Legal has not signed off on the agent issuing a rate confirmation without a human approval step.
- Two of the three engineers roll off to the carrier-portal migration in five weeks, whatever you decide.
Here is the tool definition and the handler, as currently written.
# halcyon/deskmate/tools/tender.py
TENDER_SCHEMA = {
"name": "tender_load",
"description": "Tender a load to a carrier and issue a rate confirmation.",
"parameters": {
"type": "object",
"properties": {
"broker_id": {"type": "string"},
"load_id": {"type": "string"},
"carrier_scac":{"type": "string"},
"rate_cents": {"type": "integer"},
"max_rate_cents": {"type": "integer"}, # optional
},
"required": ["broker_id", "load_id", "carrier_scac", "rate_cents"],
},
}
def tender_load(broker_id, load_id, carrier_scac, rate_cents,
max_rate_cents=None):
if not user_can_tender(broker_id):
raise PermissionError("not authorized")
if max_rate_cents and rate_cents > max_rate_cents:
raise ValueError("rate above ceiling")
return tms.post("/tenders", {
"broker_id": broker_id, "load_id": load_id,
"carrier_scac": carrier_scac, "rate_cents": rate_cents,
})
Write the readiness bar and make the call.
Constraints to consider
- You cannot retrain the model or change vendors. You can change the prompt, the scaffolding, the tools, the retrieval, the guardrails, and the interface.
- The $1,500/month budget is fixed for the quarter. Finance will not revisit it.
- Whatever ships must be operable by one engineer after week five.
- Any capability that issues a binding rate confirmation without human approval requires documented legal sign-off before launch. No launch-then-fix, no parallel-track sign-off.
- You will present to a leadership group that includes Halcyon’s VP of Operations, who thinks handing tendering authority to a language model is reckless.
AI Usage Guidance
We expect you to use AI tools. We evaluate how you use them — not whether you use them. Evidence of iteration, redirection, and critical evaluation scores higher than a polished output with no process documentation.
The single highest-signal indicator: your video answer to the mandatory AI question. If you cannot name a specific moment where you redirected AI output, evaluators will assume you did not.
Mandatory AI question for your video: Walk me through one moment where you disagreed with, pushed back on, or redirected what the AI gave you — and what you did instead. Name the specific moment. Explain what the AI produced that didn’t meet the bar, what you did differently, and why.
Speak naturally. Communication is assessed on clarity of technical ideas and logical structure — not verbal polish, accent, or filler words.
What You'll Accomplish
Define launch readiness for an agent that takes consequential actions, with thresholds rather than adjectives
Audit an agent tool interface for authorization and idempotency defects
Design guardrails that are mechanisms in code rather than statements of intent
Reason explicitly about the capability / latency / cost tradeoff with real numbers against a fixed budget
Defend a go/no-go decision to a leadership audience that includes an informed skeptic
How Your Work Will Be Scored
What to Submit
File 1 — Launch Readiness Brief
Format: .pdf, .doc, .docx, .rtf, .txt, .md
Your readiness criteria with explicit numeric thresholds, your go / limited-GA / no-go recommendation, and the five-week plan behind it.
Sign in to upload files
File 2 — README Document
Format: .pdf, .doc, .docx, .rtf, .txt, .md
Three required sections:
- Section A — Written analysis: the tradeoff argument, with the numbers attached. Include how you get from $3,690/month to $1,500/month, or why you would not launch at that volume. State what you are trading against what. 350–450 words.
- Section B — The tool audit, the guardrails, and the rollback. Three parts:
- (i) Tool audit. Review the tender_load schema and handler above. Name every defect you find and rewrite the parts that need rewriting — as code, not prose. Say which defect you consider most severe and why.
- (ii) Readiness and guardrails. The eval suite you would run before launch: what the task suite measures, roughly how big, how success is judged per case, and the threshold that gates the launch. Then: the one failure mode you consider unacceptable at any rate, and the guardrail that prevents it. The guardrail must be a mechanism.
- (iii) Rollback. The trigger (numeric), the owner (a role), and how long it takes to execute. Say what happens to tenders already issued when it fires.
- Section C — AI Usage Log (Mandatory): This is not a trick. We want to see how you work with AI — not whether you used it. In a short section of your README, document your AI collaboration process. For each significant interaction with an AI tool, briefly note: what you asked the AI to help with / what it gave you / what you kept, changed, or rejected — and why. Three interactions documented is sufficient. The log does not need to be exhaustive.
Sign in to upload files
File 3 — Video Walkthrough
Format: .mp4, .mov, .webm
Record as MP4 or MOV and upload directly on the Provn platform as a separate file. Cover: (1) your recommendation and the readiness bar, ~60 sec; (2) the tool audit — walk the code and show what you changed, ~2 min; (3) the eval suite and the gating threshold, ~90 sec; (4) the cost and latency tradeoff with the numbers, ~60 sec; (5) how you answer the VP of Operations, ~60 sec; (6) the mandatory AI question, ~60 sec; plus ~30 sec for what you would do differently with more time. Speak naturally, as if briefing your leadership team directly — technical clarity matters, verbal polish does not.
Sign in to upload files
Create a free account to upload your work. Your progress saves as a draft until you submit.
On this page