Fixing a Faulty AI Travel Response
Identify and correct an inaccurate AI chat response to a customer
Create a free account to upload your work. Your progress saves as a draft until you submit.
What You'll Be Doing
Scenario — "The Refund Mix-Up"
Context: Axel's AI chat agent helps travelers understand their flight options including when they're eligible for a refund or credit.
A customer sent this message: "Hi, my flight was canceled by the airline. Can I get my money back?"
The AI responded: "Since your ticket is non-refundable, you are not eligible for a refund, but you can use the credit for future travel."
The response was incorrect — when an airline cancels a flight, passengers are entitled to a refund, even on non-refundable tickets.
Your Task:
You've been asked to:
- Find what went wrong — what about the prompt or rule likely caused the incorrect reply?
- Fix it — write or describe an updated prompt snippet or instruction that would lead the model to give the right answer
- Test it — show what the correct AI reply should look like for the same user message
- Explain how you'd validate the fix before rolling it out (e.g., sample test chats, review process)
Submission Guidelines
Video Requirements:
Your 7-10 minute video must include ALL three components:
1. Introduction (1-2 minutes)
- Share one example of a time you improved how information was communicated — by a person, a system, or a process. What changed as a result? We want to know how you think.
2. Traits Assessment (2-3 minutes)
Which situation sounds more like the kind of work that energizes you — and why?
- Scenario A — "The Thursday Fix" (Execution): You notice the AI agent is giving wrong answers about baggage fees for one airline. You quickly verify the correct rule, update the prompt instructions, and confirm improved responses in testing. Would you enjoy that kind of direct, fast-turnaround problem solving?
- Scenario B — "The Pattern Builder" (Discovery): You see that refund-related mistakes are happening across multiple airlines. You spend time reviewing transcripts, identifying the root cause, and proposing a new set of logic or classification improvements. Would you prefer diagnosing larger patterns and designing fixes that scale?
There's no right or wrong answer — just tell us which type of challenge fits how you work best.
3. Challenge Response (3-5 minutes)
Walk us through how you'd fix the faulty AI response in the scenario:
- What's wrong with the AI's answer?
- How would you adjust the prompt or rule?
- How would you test that it now works reliably?
Keep it simple and practical — focus on your thought process and clarity.
AI Usage Declaration:
At the end of your video, please state:
- ☐ I did not use any AI tools
- ☐ I used AI tools (briefly describe how, including what I changed or adapted)
Challenge Guidelines
- Video: 7–10 minutes (continuous recording, one take)
- Written Artifacts: Two short documents (max 1 page each)
- Scenario: Identify and correct an inaccurate AI chat response to a customer
- AI Usage Declaration: Required
What You'll Accomplish
Diagnose AI response quality issues
Validate and test AI behavior improvements
Balance accuracy with user experience Monitor and improve operational quality
Monitor and improve operational quality
Write clear prompt instructions and guidelines
How Your Work Will Be Scored
What to Submit
Required Submissions:
- Prompt Fix & Example Output (Required)
- Format: PDF or Document
- Length: 1 page maximum
- Show your corrected prompt or guideline text
- Include an example of the right AI reply the user should see
- Briefly explain what changed and why
- Quality Assurance Plan (Required)
- Format: PDF or Document
- Length: 1 page maximum
- Describe how you'd:
- Test the fix on a few similar chat examples
- Monitor performance once deployed (e.g., error rate, customer feedback)
- Share insights with engineering or training teams for longer-term improvements
- Keep it lightweight — this is about how you think, not formal testing scripts
- Video Recording (Required)
- Format: MP4
- Length: 7-10 minutes (continuous recording, one take)
- Must include all three components detailed below
Submission Guidelines
Video Requirements:
Your 7-10 minute video must include ALL three components:
1. Introduction (1-2 minutes)
- Share one example of a time you improved how information was communicated — by a person, a system, or a process. What changed as a result?
2. Traits Assessment (2-3 minutes)
Which situation sounds more like the kind of work that energizes you — and why?
- Scenario A — "The Thursday Fix" (Execution): You notice the AI agent is giving wrong answers about baggage fees for one airline. You quickly verify the correct rule, update the prompt instructions, and confirm improved responses in testing. Would you enjoy that kind of direct, fast-turnaround problem solving?
- Scenario B — "The Pattern Builder" (Discovery): You see that refund-related mistakes are happening across multiple airlines. You spend time reviewing transcripts, identifying the root cause, and proposing a new set of logic or classification improvements. Would you prefer diagnosing larger patterns and designing fixes that scale?
There's no right or wrong answer — just tell us which type of challenge fits how you work best.
3. Challenge Response (3-5 minutes)
Walk us through how you'd fix the faulty AI response in the scenario:
- What's wrong with the AI's answer?
- How would you adjust the prompt or rule?
- How would you test that it now works reliably?
Keep it simple and practical — focus on your thought process and clarity.
AI Usage Declaration:
At the end of your video, please state:
- ☐ I did not use any AI tools
- ☐ I used AI tools (briefly describe how, including what I changed or adapted)
Prompt Fix & Example Output
Format: .pdf, .doc, .docx, .rtf, .txt, .md
PDF or Document. Show your corrected prompt or guideline text, include an example of the correct AI reply, and briefly explain what changed and why.
Sign in to upload files
Quality Assurance Plan
Format: .pdf, .doc, .docx, .rtf, .txt, .md
PDF or Document. Describe how you would test the fix, monitor performance once deployed, and share insights with teams for improvements.
Sign in to upload files
Video Recording
Format: .mp4, .mov, .webm
MP4 format. Introduction (1-2 min), traits assessment (2-3 min), and challenge response (3-5 min). Include AI usage declaration at the end.
Sign in to upload files
Create a free account to upload your work. Your progress saves as a draft until you submit.
On this page