Northwind AI's support team was answering the same handful of questions dozens of times a day — questions their own documentation already covered, if customers could find it. We built a retrieval-augmented support assistant grounded in their existing knowledge base, with a visible, honest handoff to a human whenever it wasn't confident, rather than a chatbot pretending to know things it didn't.
Northwind AI's support team was drowning in repetitive tickets that their own documentation already answered.
Every repetitive ticket the support team answered manually was time not spent on the harder, higher-value problems only a human could solve — and customers were waiting longer for answers documentation already had.
B2B SaaS — Northwind AI's own support team costs scale with its customer base, making support efficiency a direct margin lever, not just a cost center to minimize.
AI-powered customer support tooling, a fast-moving category where trust and honesty about AI limitations increasingly differentiate products from support chatbots with a reputation for confidently wrong answers.
A traditional ticketing system and a documentation knowledge base, with no existing AI or automation in the support workflow.
Competitors were shipping AI support chatbots quickly, but several had already damaged customer trust with overconfident, incorrect answers — a cautionary example Northwind AI wanted to avoid repeating.
Categorized three months of support tickets to quantify how many were genuinely repetitive questions documentation already answered.
Audited the existing knowledge base for gaps, outdated content, and inconsistent structure before treating it as a retrieval source.
Ran workshops with support agents to understand which questions they trusted an assistant to answer, and which needed to always stay human.
Use retrieval-augmented generation grounded in existing docs, not a fine-tuned model.
Reasoning
Northwind AI's documentation changes weekly; RAG could reflect updates immediately, while fine-tuning would require retraining.
Trade-off
Answer quality depends directly on documentation quality, so the documentation audit became a prerequisite, not an afterthought.
Always visibly hand off to a human when the assistant's confidence was low.
Reasoning
An assistant that hallucinates confidently is worse than one that admits uncertainty — trust mattered more than resolution rate.
Trade-off
Lower auto-resolution rate than a more aggressive confidence threshold would have produced.
Exclude billing and account security questions from automated answers entirely.
Reasoning
The cost of a wrong answer in these categories was too high relative to the time saved.
Trade-off
Left some genuinely repetitive tickets in these categories still requiring a human response.
Customer
Interacts via a support widget embedded in the product, or an existing support ticket.
What happened
Outdated documentation sections caused the assistant to confidently repeat incorrect information.
How we resolved it
Added a documentation freshness score that down-weighted stale content in retrieval, alongside a parallel effort to update the most-retrieved outdated pages.
What happened
The initial confidence threshold for human handoff was miscalibrated, escalating too many easy questions.
How we resolved it
Recalibrated the threshold using real conversation outcome data from the first two weeks of limited launch.
Approach
We built a retrieval-augmented assistant grounded in their existing knowledge base, with a clear, visible handoff to a human whenever it wasn't confident.
Outcome
First-response time dropped sharply without any reduction in resolution quality.
First-response time
Tickets auto-resolved
Customer complaints about support answers
Proactive suggestions surfaced to support agents mid-conversation for currently-excluded categories like billing
Byld IQ recommendationExpanding retrieval to product usage data, not just documentation
Client planA self-service documentation gap report so the support team can prioritize what to write next
Byld IQ recommendationBuildPath turns your version of this problem into a personalized roadmap in about three minutes — or talk to Byld first if you still have questions.