AI / Agentic Software Engineer

Synthesized from ten live postings at Anthropic, OpenAI, Google, Microsoft AI, Amazon, NVIDIA, Databricks, Stripe, Scale AI, and Sierra, read from each employer's own careers board on August 9, 2026. Not any one company's job description; the center of mass.

Your correctness contract changed from deterministic to statistical, and nobody rewrote your job description to say so. The thing you deployed can take actions on a customer's behalf, spends a latency budget in tokens instead of milliseconds, and is wrong at a rate you are expected to know.

You work with researchers and ML engineers as first-class partners, expected to build the tools they use rather than only consume the model they ship. You work with product and design, and at Sierra and Scale AI directly with customers. The stated floor is around five years of professional engineering, but the spread runs from 2+ at Stripe to 8+ at NVIDIA, and the real gate has not changed: distributed systems, scalable APIs, cloud-native infrastructure, and evidence you have shipped something model-powered and kept it running.

Ten shifts show up across the postings. Each is difficult to fake on video and impossible to fake in code, which is why the ladder is built to surface them.

Every agent framework a candidate would put on a resume appears in zero of ten postings. Every eval platform: zero. Every branded vector database: zero. What employers name instead is languages (Python in eight, TypeScript in six), clouds, containers, and observability. NVIDIA asks for experience with “multiple agent frameworks and agent harnesses,” deliberately plural and unbranded, in a posting whose whole job is building those frameworks. The one named interop technology in the sample is MCP, an open protocol, mentioned once.

So no Provn rubric for this role awards points for naming a framework, a vector store, or an eval platform. We score the code and the reasoning. Did you bound the loop? Validate the model's output before acting on it? Separate the check a machine can run from the one a human has to? Twelve lines of validation with a stated failure budget outscore four frameworks and an unvalidated json.loads.

• Anthropic

  $320K to $625K

  Staff engineer building AI developer tools; expert React expected alongside agent harness work. Excitement about AI-assisted coding is a stated requirement.

  Tier 1

• OpenAI

  $230K to $385K + equity

  Codex core agents: optimize tokens, latency, reliability, and cost across the production fleet, in one bullet. Sandboxed execution required.

  Tier 1

• Sierra

  $230K to $390K

  The only employer whose engineering titles are built around the word Agent: Agent, Agent Builder, Agent Architecture, Agent Data Platform.

  Tier 1

• NVIDIA

  $184K to $356.5K

  Asks for experience with multiple agent frameworks and harnesses, plural and unbranded, in a role that builds them.

  Tier 1

• Google

  $147K to $211K + bonus + equity

  Applied AI; names LLM orchestration frameworks as a category without naming one; the only posting with an eval hard requirement, via ML-infra minimums.

• Microsoft AI

  $142.8K to $304.2K

  Principal engineer on AI platform and agent execution; sandboxing preferred; researchers named as counterparts.

  Tier 2

• Amazon / AWS

  $184.9K to $250.2K

  AgentCore: managed agent loops, tool orchestration, memory and context management in the body. “5+ years of software development” in the bar. Verified high volume: 194 agentic-AI SDE hits.

  Tier 2

• Databricks

  $166K to $225K

  Staff backend on the AI platform. A Staff band paying below Amazon's Senior band.

  Tier 2

• Stripe

  CA$172K to CA$312K

  ML infrastructure and platform; hands-on use of AI tools to accelerate your work is a required qualification.

  Tier 2

• Scale AI

  $179.4K to $224.25K

  Enterprise gen-AI product engineering; customer contact is a stated qualification.

  Tier 2

Provn is not affiliated with any employer listed. Descriptions summarize each company's public job posting as read in August 2026.

1. Practice20 min total

   What Breaks in Production

   Forty lines of LLM-calling code. Name the distinct classes of thing that will fail, rank them, and write the check that would catch the worst one.

   Isolates: Can you read forty lines of LLM-calling code and name the distinct classes of thing that will fail? 3 to 4 min video.

2. Easy28 min total

   The 9% That Costs Money

   Deskmate extracts loads correctly 91% of the time and no confidence score exists. Design the gate around it, with a stated failure budget, and write it.

   Isolates: Can you design the gate around a component that is right 91% of the time, when no confidence score exists? 4 to 5 min video.

3. Medium40 min total

   The Agent Got Slower and Dumber

   Metrics plus code for an agent loop that is degrading. Diagnose it, propose the diff, and build the regression gate that would have caught it.

   Isolates: Can you diagnose a degrading agent loop from metrics plus code, and build the regression gate that would have caught it? 5 to 6 min video.

4. Hard40 min total

   The Agent That Can Spend Money

   Deskmate can now commit a quote. Set the readiness bar, find the authorization bug, and defend a launch call with the cost arithmetic intact.

   Isolates: Can you set the readiness bar, find the authorization bug, and defend a launch call with the cost arithmetic intact? 6 to 8 min video.

Every tier is scored on the same five dimensions. Practice, Easy and Medium also contain a short, clearly marked edge-case block you are asked to complete without AI assistance; enumerating edge cases from first principles is the fastest read on whether someone has operated a system like this.

What Transformative looks like

On the Practice tier the core dimension is failure-class diagnosis of the supplied code. Transformative separates distinct failure classes (an unvalidated model output acted on directly, an unbounded loop, an injection path through customer text, a cost path with no ceiling) and ranks them by consequence, not by how often they would fire. Adoptive names at least two classes and picks the right first fix with a reason; that is the advance bar. Capable lists real bugs but treats them as one “the LLM is unreliable” problem. Below Capable names one failure class correctly but misses the injection path and the cost path entirely. Not Yet proposes a bigger model or a better prompt as the fix.

Communication is judged on whether the person you are explaining to could act on your video alone. Never on accent, pace, or filler words.

1. The extractor is right 91% of the time and there is no confidence score. How do you ship it?

   Design the gate, not the prompt. Validate structure and business rules on every output, route anything that fails to a human queue, and state the failure budget you are accepting on what passes. The strong answer says which checks a machine runs, which a person runs, and what the queue costs per week.

   Where it comes from: Amazon and OpenAI put managed agent loops and reliability in the engineer's scope; six of ten postings mention evaluations.

2. Walk me through how you would debug an agent that has gotten slower and less accurate over six weeks with no code change.

   Start with what did change: the model version, the prompt, the retrieval corpus, the tool responses, the input distribution. Read trajectories, not stack traces. The strong answer names the metric that moved first and builds the regression check that would have flagged it.

   Where it comes from: Microsoft, Databricks, Stripe, Amazon and Google name monitoring, tracing and incident response as owned responsibilities.

3. A retrieval change from top-3 to top-10 chunks improves answer quality. Do you ship it?

   Only with the cost math. State the token delta per request, the monthly figure at current volume, and the quality gain in a measurable form. If you cannot measure the gain, you are not making an engineering decision.

   Where it comes from: OpenAI: optimize tokens, latency, reliability, and cost. Six of ten postings make cost an owned responsibility.

4. The agent can now place an order. What does the authorization model look like?

   Least privilege per action, a confirmation step for anything irreversible, a spend ceiling enforced outside the model, and an audit record of what the agent did and why. The strong answer treats the auth model as the product and names the bug class it prevents.

   Where it comes from: Amazon names identity alongside networking and observability; four of ten require sandboxing where the agent runs code.

5. Tell me about a time an AI coding tool gave you something wrong and what you did.

   Name the specific moment, what it produced, what was wrong against the bar you were holding, and what you shipped instead. This is also the mandatory AI question in every Proving Ground video.

   Where it comes from: Stripe requires hands-on use of AI tools to accelerate your work; Anthropic requires enthusiasm for AI-assisted coding.