AI Experience Engineer Challenge
Senior AI Experience Engineer Application Challenge-- Designing an Intelligent Workflow Assistant
Create a free account to upload your work. Your progress saves as a draft until you submit.
What You'll Be Doing
This challenge assesses your ability to design and implement an AI-powered workflow assistant for PortX, a company that powers integrations between banks, credit unions, and fintech companies.
Scenario — "AI Workflow Assistant for Integration Setup":
PortX powers integrations between banks, credit unions, and fintechs. New customers often ask: "How do I connect my banking core to a fintech partner for payments or onboarding?" Right now, this setup process involves reading documentation and configuring API endpoints manually.
Your task is to design an AI Workflow Assistant that can guide users through connecting a new integration safely, interactively, and with human-level clarity.
Record a video walking us through your approach, and submit two short design documents outlining your system architecture and reliability strategy.
Challenge Guidelines
- Video: 7–10 minutes (continuous recording, one take)
- Written Artifacts: Two short (1 page max each) documents
- Scenario: Design an AI agent that guides users through API integration setup
- AI Usage Declaration: Required
What You'll Accomplish
Design orchestration layers for LLM-powered agents
Implement safety guardrails and human-in-the-loop workflows
Build transparent, explainable AI interactions
Balance reliability with flexibility in agentic systems
Create audit trails for compliance and trust
How Your Work Will Be Scored
What to Submit
Required Submissions:
1. System Design Sketch (Required)
- Format: PDF or Document
- Length: 1 page maximum
- Content: Provide a simple diagram or outline showing:
- The main components (agent logic, data connectors, model orchestration)
- How the system handles retrieval, planning, and action execution
- How it maintains safety and traceability (logs, approval prompts, or tool permissions)
Focus on architecture and reasoning — not full code. A hand-drawn diagram is acceptable if clear.
Strong Example: "I'd design an orchestration layer with LangChain's agent executor and custom PortX tools: ListPartnersTool for available integrations, SetupEndpointTool for authenticated API setup. Each step prompts user confirmation ('I'll create this endpoint — okay?') and logs it in Datadog. Guardrails: restrict API actions to read-only until confirmation; use retry on timeout. Success metric: 90% of users complete setup with zero support tickets."
Weak Example: "Use GPT to generate configs and push them to the API automatically."
2. Reliability & Feedback Plan (Required)
- Format: PDF or Document
- Length: 1 page maximum
- Content: Describe how you'd ensure high-quality, trustworthy AI outputs in production:
- What guardrails or fallback strategies would you use (e.g., model confidence thresholds, rule-based checks, human-in-the-loop)?
- How would you capture and use user feedback to improve performance over time?
- How would you measure success — accuracy, completion rate, latency, or user trust?
Think about both user experience and system health.
3. Video Recording (Required)
- Format: MP4
- Length: 7-10 minutes (continuous recording, one take)
- Must include all three components detailed below
Submission Guidelines
Video Requirements:
Your 7-10 minute video must include ALL three components:
1. Introduction (1-2 minutes)
Briefly describe one AI-powered experience you've built or prototyped. What made it effective for users, and how did you validate its performance?
2. Traits Assessment (2-3 minutes)
Which of these situations fits your natural working style better — and why?
Scenario A — "The Launch Sprint" (Execution): You're asked to ship an AI feature that summarizes customer feedback from support tickets. The models and APIs are known — your challenge is tuning prompts, handling edge cases, and delivering it into production by the end of the week. Would that kind of focused, high-velocity delivery excite you?
Scenario B — "The Discovery Loop" (Exploration): You're asked to design a new "AI Workflow Assistant" that helps financial institutions connect data pipelines to fintech partners. No one knows exactly how it should work yet — you'll explore, prototype, and define the core interaction model. Would you enjoy that kind of open-ended, product-defining work?
There's no right or wrong answer — we just want to understand how you like to work.
3. Challenge Response (3-5 minutes)
Walk us through how you would design and implement the AI Workflow Assistant described above:
- How would you structure the agent or LLM orchestration?
- How would you connect it safely to PortX APIs or partner data?
- How would you make it reliable, explainable, and user-friendly?
AI Usage
At the end of your video, please tell us if you used AI. If you did, let us know how you used it.
System Design Sketch
Format: .pdf, .doc, .docx, .rtf, .txt, .md
Provide a simple diagram or outline showing:
- The main components (agent logic, data connectors, model orchestration)
- How the system handles retrieval, planning, and action execution
- How it maintains safety and traceability (logs, approval prompts, or tool permissions)
Focus on architecture and reasoning — not full code. A hand-drawn diagram is acceptable if clear.
Strong Example: "I'd design an orchestration layer with LangChain's agent executor and custom PortX tools: ListPartnersTool for available integrations, SetupEndpointTool for authenticated API setup. Each step prompts user confirmation ('I'll create this endpoint — okay?') and logs it in Datadog. Guardrails: restrict API actions to read-only until confirmation; use retry on timeout. Success metric: 90% of users complete setup with zero support tickets."
Weak Example: "Use GPT to generate configs and push them to the API automatically."
Sign in to upload files
Reliability & Feedback Plan
Format: .pdf, .doc, .docx, .rtf, .txt, .md
Describe how you'd ensure high-quality, trustworthy AI outputs in production:
- What guardrails or fallback strategies would you use (e.g., model confidence thresholds, rule-based checks, human-in-the-loop)?
- How would you capture and use user feedback to improve performance over time?
- How would you measure success — accuracy, completion rate, latency, or user trust?
Think about both user experience and system health.
Sign in to upload files
Video Recording
Format: .mp4, .mov, .webm
Record a continuous video in one take covering all three components:
1. Introduction (1-2 minutes)
Briefly describe one AI-powered experience you've built or prototyped. What made it effective for users, and how did you validate its performance?
2. Traits Assessment (2-3 minutes)
Which of these situations fits your natural working style better — and why?
Scenario A — "The Launch Sprint" (Execution): You're asked to ship an AI feature that summarizes customer feedback from support tickets. The models and APIs are known — your challenge is tuning prompts, handling edge cases, and delivering it into production by the end of the week. Would that kind of focused, high-velocity delivery excite you?
Scenario B — "The Discovery Loop" (Exploration): You're asked to design a new "AI Workflow Assistant" that helps financial institutions connect data pipelines to fintech partners. No one knows exactly how it should work yet — you'll explore, prototype, and define the core interaction model. Would you enjoy that kind of open-ended, product-defining work?
There's no right or wrong answer — we just want to understand how you like to work.
3. Challenge Response (3-5 minutes)
Walk us through how you would design and implement the AI Workflow Assistant described above:
- How would you structure the agent or LLM orchestration?
- How would you connect it safely to PortX APIs or partner data?
- How would you make it reliable, explainable, and user-friendly?
AI Usage Declaration
At the end of your video, please tell us if you used AI. If you did, let us know how you used it.
Sign in to upload files
Create a free account to upload your work. Your progress saves as a draft until you submit.
On this page