Skip to content
All case studies
In flightPriority: UrgentSep 2026 – Dec 2026 (12 weeks / 3 releases)

AI-Powered Food Order Support Assistant

Support agents were drowning in manual lookups across five systems for every ticket. Delivered a human-in-the-loop AI assistant — context aggregation, PII masking, grounded 3-sentence summaries, categorized next-best-actions — with agents always owning the final decision.

My role
Technical Product Owner & Scrum Master — wrote the functional + non-functional requirements, drove MoSCoW + WSJF sequencing, owned GDPR sign-off as a release gate, facilitated Three Amigos and retros
Team
Technical Product Owner / Scrum Master (author) · Backend / API gateway · AI engineering · QA · Compliance · Support operations
Stack
Enterprise GPT API (zero-retention tier) · React / Vue SPA agent portal · REST integrations (5 systems) · OAuth 2.0 + RBAC · PostgreSQL audit log · Adapter pattern + circuit breakers
  • −30%

    AHT reduction

    vs baseline (target)

  • +10 pts

    FCR improvement

    vs baseline (target)

  • ≥ 70%

    Recommendation acceptance

    Approved without modification

  • ≤ 2%

    Hallucination rate

    QA audit sampling

01

The Problem

Support teams were handling thousands of food-order inquiries every week. Before responding, agents manually reviewed order details, delivery status, payment information, and prior interactions across five separate systems. The result was long handling times, inconsistent resolutions, and lower CSAT — with the stated business target being a 30% reduction in AHT and an FCR improvement.

Building an AI assistant is the obvious answer; doing it without leaking PII, hallucinating policy, or removing human accountability is the hard part. The assessment defined the MVP that ships in three months while keeping agents in full control of every customer-facing decision.

  • Manual context gathering across five systems was the single largest AHT driver
  • Existing solutions either skip PII protection or auto-execute customer-facing actions
  • Outdated delivery status silently corrupts downstream refund recommendations
  • Baseline AHT / FCR, PII field list, and AI provider policy all undecided at kickoff
02

The Architecture

Every inquiry becomes a case. The platform fans out to five REST APIs in parallel, masks PII before any external transmission, and sends a minimized payload to an enterprise GPT API on a zero-retention tier. The model returns a grounded 3-sentence summary and a categorized next-best-action — never an executed decision. Agents validate, edit, or override every recommendation in a unified web portal.

  • Agent Portal SPA — summary card on top, categorized NBA below, raw-context toggle always one click away for verification.
  • API Gateway — OAuth 2.0 + RBAC at the edge, throttling, tracing, and circuit breakers per integration.
  • Aggregation Service — parallel fan-out across ordering, delivery, payment/refund, CRM and support APIs, normalized into one schema with per-API timeout and graceful fallback.
  • Data Sanitizer — regex / NLP masking layer replacing names, emails, phone numbers, addresses, and payment references with [TOKEN] before any payload leaves the boundary.
  • Generative AI Middleware — versioned prompt templates, JSON schema enforcement for the six allowed NBA categories, confidence scoring, automatic fallback on schema violation or timeout.
  • Feedback & Audit Service — append-only, tamper-evident log of every approve / reject / override event feeding both the operational dashboard and the eval harness.
  • ISO 8601 timestamps + freshness guards (≤ 5 min since last update) — stale data blocks refund recommendations before they render.
APIDataAuthTimeoutFailure handlingRisk
OrderingItems, value, restaurant, placed timeOAuth 2.02sUnavailable panel + retry; snapshot dataMed
Delivery TrackingStatus, courier, ETA, last-update tsOAuth 2.02sLast-known-good + STALE badge; ≥ 5 min blocks refund NBAHIGH
Payment / RefundStatus, refund status, referencemTLS3sMasked references only; refund NBA disabled on failure; never raw PANHIGH
CRMLast 5 interactions, notes, segmentsOAuth 2.01sNo-history state; informational only — never a fact sourceLow
Customer SupportCase data, prior ticketsOAuth 2.02sLocal queue + async retry on write failureMed
03

The Execution

Three monthly releases gated by MoSCoW + WSJF. Sprint boards were strictly Must-Have-only; anything else was refined for the next sprint. Fully automated refund / cancellation execution was explicitly Won't-Have to preserve the human-in-the-loop constraint.

  • R1 — Foundation (W1–W4): API gateway, adapters for all five systems, unified JSON schema, normalization + timeout fallbacks, freshness guard v1, compliance spike on masking architecture.
  • R2 — AI Engine & Core UI (W5–W8): PII masking layer compliance-approved, LLM integration on the zero-retention tier, prompt engineering + eval harness, React portal with summary card, NBA, raw-context toggle.
  • R3 — Feedback & Go-Live (W9–W12): override / feedback capture, immutable audit log, KPI dashboard, UAT with 10–20 pilot agents, GDPR audit, production deploy + hypercare.
  • Three Amigos sessions (PO, QA Lead, Tech Lead) before each sprint planning to map edge cases in the data layer before writing code.
  • Critical defect (outdated delivery status) was logged as the highest-priority bug + impediment and swarmed with paired dev + QA until regression coverage landed.
  • Stakeholder dashboard request mid-sprint was logged to backlog, declined for the current sprint, and explicitly targeted post-MVP to protect the goal.
04

The Impact

The release removes the single largest AHT driver — manual context gathering across five systems — and standardizes decision quality across agents, while preserving the GDPR posture and keeping humans in charge of every customer-facing action.

  • Business KPI — AHT reduction ≥ 30% vs baseline; FCR improvement ≥ +10 pts vs baseline.
  • Operational KPI — portal uptime ≥ 99.9%, gateway latency ≤ 300ms p95, third-party timeout rate ≤ 1%.
  • AI Quality KPI — recommendation acceptance ≥ 70% (approved unmodified), hallucination rate ≤ 2% (QA sampling), summary adherence ≥ 95% (grounded, ≤ 3 sentences).
  • Compliance KPI — zero GDPR incidents, append-only audit retained ≥ 12 months, DPIA signed before go-live.
  • Adoption KPI — ≥ 80% of pilot agents daily-active by W12; ≥ 90% feedback / override completion on rejections.

Fallback behavior matrix

  • Delivery data stale (≥ 5 min) — STALE badge; refund NBA suppressed; 'Data stale — escalate manually' shown.
  • Payment API unavailable — refund NBA disabled; 'Payment status unknown — escalate' shown; non-refund actions proceed.
  • AI confidence low — caution label + raw-context toggle highlighted; agent decision still possible.
  • Required data missing — 'Unavailable' panel + retry; AI prompted only with what exists; summary notes the gap.
  • Systems disagree — source-of-truth hierarchy applied (ordering / delivery / payment over CRM); conflict surfaced to the agent explicitly.
  • AI service outage / timeout — circuit breaker opens; portal switches to manual-template mode with raw context; requests replayed async. Agent is never blocked.

Top risks and mitigations

  • Outdated delivery status — freshness guard + read-time validation + regression tests (TPO + Eng).
  • AI hallucination from misread / missing data — masked-but-complete context, strict JSON schemas, confidence scores, rules validation (AI Eng).
  • GDPR / payment-reference exposure — tokenization pre-LLM, zero-retention DPA, audit logs, DPIA before go-live (Compliance).
  • Agent adoption resistance — shadowing sessions, raw-context toggle, feedback loop, agent champions (TPO + Ops).

AI model decision

  • Recommended: Option A — External enterprise GPT API on a zero-retention tier, protected by a backend PII masking / tokenization layer and a signed DPA.
  • Rationale: 3-month deadline makes internal LLM deployment unfeasible; zero-retention keeps GDPR posture defensible.
  • Exit path: model-abstraction layer so a future swap to an internal LLM is configuration, not a rewrite.

Source: TPO / SM Assessment — AI-Powered Food Order & Customer Support Assistant, Yamama, Sep 2026