Challenges/Gordian Software/Operations Manager/Fixing a Faulty AI Travel Response

    Fixing a Faulty AI Travel Response

    Identify and correct an inaccurate AI chat response to a customer

    Operations
    AI Quality Assurance
    Prompt Engineering
    Process Improvement
    Customer Support
    Estimated Time:
    30 minutes
    Difficulty:Intermediate
    Status:Not started
    Start this challenge

    Create a free account to upload your work. Your progress saves as a draft until you submit.

    What You'll Be Doing

    Scenario — "The Refund Mix-Up"

    Context: Axel's AI chat agent helps travelers understand their flight options including when they're eligible for a refund or credit.

    A customer sent this message: "Hi, my flight was canceled by the airline. Can I get my money back?"

    The AI responded: "Since your ticket is non-refundable, you are not eligible for a refund, but you can use the credit for future travel."

    The response was incorrect — when an airline cancels a flight, passengers are entitled to a refund, even on non-refundable tickets.

    Your Task:

    You've been asked to:

    1. Find what went wrong — what about the prompt or rule likely caused the incorrect reply?
    2. Fix it — write or describe an updated prompt snippet or instruction that would lead the model to give the right answer
    3. Test it — show what the correct AI reply should look like for the same user message
    4. Explain how you'd validate the fix before rolling it out (e.g., sample test chats, review process)

    Submission Guidelines

    Video Requirements:

    Your 7-10 minute video must include ALL three components:

    1. Introduction (1-2 minutes)

    • Share one example of a time you improved how information was communicated — by a person, a system, or a process. What changed as a result? We want to know how you think.

    2. Traits Assessment (2-3 minutes)

    Which situation sounds more like the kind of work that energizes you — and why?

    • Scenario A — "The Thursday Fix" (Execution): You notice the AI agent is giving wrong answers about baggage fees for one airline. You quickly verify the correct rule, update the prompt instructions, and confirm improved responses in testing. Would you enjoy that kind of direct, fast-turnaround problem solving?
    • Scenario B — "The Pattern Builder" (Discovery): You see that refund-related mistakes are happening across multiple airlines. You spend time reviewing transcripts, identifying the root cause, and proposing a new set of logic or classification improvements. Would you prefer diagnosing larger patterns and designing fixes that scale?

    There's no right or wrong answer — just tell us which type of challenge fits how you work best.

    3. Challenge Response (3-5 minutes)

    Walk us through how you'd fix the faulty AI response in the scenario:

    • What's wrong with the AI's answer?
    • How would you adjust the prompt or rule?
    • How would you test that it now works reliably?

    Keep it simple and practical — focus on your thought process and clarity.

    AI Usage Declaration:

    At the end of your video, please state:

    • ☐ I did not use any AI tools
    • ☐ I used AI tools (briefly describe how, including what I changed or adapted)

    Challenge Guidelines

    • Video: 7–10 minutes (continuous recording, one take)
    • Written Artifacts: Two short documents (max 1 page each)
    • Scenario: Identify and correct an inaccurate AI chat response to a customer
    • AI Usage Declaration: Required

    What You'll Accomplish

    Diagnose AI response quality issues

    Validate and test AI behavior improvements

    Balance accuracy with user experience Monitor and improve operational quality

    Monitor and improve operational quality

    Write clear prompt instructions and guidelines

    How Your Work Will Be Scored

    Creative Problem Solving & Accuracy-25%Systems Thinking & QA Design-20%High Agency / Ownership — 20% Automation & AI Fluency-15%Customer Empathy & Communication-15%AI Usage Transparency-5%

    What to Submit

    Required Submissions:

    1. Prompt Fix & Example Output (Required)
      • Format: PDF or Document
      • Length: 1 page maximum
      • Show your corrected prompt or guideline text
      • Include an example of the right AI reply the user should see
      • Briefly explain what changed and why
    2. Quality Assurance Plan (Required)
      • Format: PDF or Document
      • Length: 1 page maximum
      • Describe how you'd:
        • Test the fix on a few similar chat examples
        • Monitor performance once deployed (e.g., error rate, customer feedback)
        • Share insights with engineering or training teams for longer-term improvements
      • Keep it lightweight — this is about how you think, not formal testing scripts
    3. Video Recording (Required)
      • Format: MP4
      • Length: 7-10 minutes (continuous recording, one take)
      • Must include all three components detailed below

    Submission Guidelines

    Video Requirements:

    Your 7-10 minute video must include ALL three components:

    1. Introduction (1-2 minutes)

    • Share one example of a time you improved how information was communicated — by a person, a system, or a process. What changed as a result?

    2. Traits Assessment (2-3 minutes)

    Which situation sounds more like the kind of work that energizes you — and why?

    • Scenario A — "The Thursday Fix" (Execution): You notice the AI agent is giving wrong answers about baggage fees for one airline. You quickly verify the correct rule, update the prompt instructions, and confirm improved responses in testing. Would you enjoy that kind of direct, fast-turnaround problem solving?
    • Scenario B — "The Pattern Builder" (Discovery): You see that refund-related mistakes are happening across multiple airlines. You spend time reviewing transcripts, identifying the root cause, and proposing a new set of logic or classification improvements. Would you prefer diagnosing larger patterns and designing fixes that scale?

    There's no right or wrong answer — just tell us which type of challenge fits how you work best.

    3. Challenge Response (3-5 minutes)

    Walk us through how you'd fix the faulty AI response in the scenario:

    • What's wrong with the AI's answer?
    • How would you adjust the prompt or rule?
    • How would you test that it now works reliably?

    Keep it simple and practical — focus on your thought process and clarity.

    AI Usage Declaration:

    At the end of your video, please state:

    • ☐ I did not use any AI tools
    • ☐ I used AI tools (briefly describe how, including what I changed or adapted)

    Prompt Fix & Example Output

    Document · 1 page maxRequired

    Format: .pdf, .doc, .docx, .rtf, .txt, .md

    PDF or Document. Show your corrected prompt or guideline text, include an example of the correct AI reply, and briefly explain what changed and why.

    Sign in to upload files

    Quality Assurance Plan

    Document · 1 page maxRequired

    Format: .pdf, .doc, .docx, .rtf, .txt, .md

    PDF or Document. Describe how you would test the fix, monitor performance once deployed, and share insights with teams for improvements.

    Sign in to upload files

    Video Recording

    Video · 7-10 minRequired

    Format: .mp4, .mov, .webm

    MP4 format. Introduction (1-2 min), traits assessment (2-3 min), and challenge response (3-5 min). Include AI usage declaration at the end.

    Sign in to upload files