Challenges/Arrivia/Software Engineer, Full Stack Engineer/Full Stack AI Engineer — Identity Risk Scorer

Full Stack AI Engineer — Identity Risk Scorer

Estimated Time:

40 minutes

Status:Not started

What You'll Be Doing

You're joining the BuilderEx team at a travel loyalty company that operates identity and authentication for multiple partner brands. The team is mid-migration to a unified OAuth 2.0/OpenID Connect identity provider — three legacy auth systems are consolidating into one, but the migration is not complete.

The problem you've been handed on your second week: The fraud team has flagged a pattern — 847 logins over the past 30 days succeeded (valid credentials, valid OIDC token issued) but showed anomalous post-auth behavior: unusual geolocation, device switches within minutes of login, then immediate redemption requests against high-value loyalty accounts.

The current system trusts a valid token completely. There is no post-authentication risk assessment.

Your manager asks you to build a risk-scoring middleware that sits in the post-authentication flow. After a valid OIDC token is issued, your middleware receives a login event payload and returns a structured risk_decision object that downstream systems use to determine whether to allow, step-up-authenticate, or block the session.

Constraints:

You cannot modify the OIDC token issuance flow — the identity provider is a managed system; your middleware hooks in after token issuance.
The explanation field must be readable by a compliance officer without translation — "risk score exceeds threshold" is not acceptable.
Scope your PoC to what's achievable in a two-week sprint — heuristic, rule-based, lightweight ML, or a combination is fine; it just needs to be functional, observable, and improvable.
Your team will own this in production, including on-call at 2am when the scorer starts generating false positives and blocking loyal platinum-tier members. Design accordingly.

AI Usage Guidance: Using AI tools is expected and encouraged — how you use AI is part of what's evaluated. Your video walkthrough must include this mandatory question: "Walk me through one moment where you disagreed with, pushed back on, or redirected what the AI gave you — and what you did instead. Name the specific moment. Explain what the AI produced that didn't meet the bar, what you did differently, and why."

What You'll Accomplish

Demonstrate ability to design a post-authentication risk scoring system that integrates with an existing OAuth 2.0/OIDC flow without modifying it

Build a risk scoring implementation that balances security signal accuracy with the real cost of false positives on legitimate users

Show production engineering judgment: code structure, error handling, and observability appropriate for an on-call ownership model

Design an API output that serves both downstream engineering systems and non-technical compliance audit requirements

Document architectural trade-offs, incident response thinking, and AI collaboration process

How Your Work Will Be Scored

IAM Architecture & SSO Execution — 27% — Middleware correctly positioned in the post-auth flow, OIDC constraint honored, latency and failure-mode trade-offs addressedAI/ML Risk Scoring Integration — 22% — Signal selection observable at runtime, explanation field serves a compliance officer, false positive cost explicitly consideredFull-Stack Production Engineering — 18% — Code structured for on-call ownership, error handling covers real failure modes, incident runbook provides actionable stepsDeveloper Experience & API Design — 13% — risk_decision output is self-documenting, Section A reads as a design document, system is extensible without a rewriteAI Fluency — 10% — AI Usage Log, Section B2 reasoning, and video AI question together demonstrate genuine judgment about when/how to use AIResume & Background — 10% — Evaluated separately from challenge artifacts

What to Submit

No submission guidelines provided.

On this page