We're still building things here! Help us improve by reporting bugs here.
Sign in

Conversations, Sessions & the Responder

How the responder assembles context, retrieves knowledge, builds prompts, runs safety checks, and returns fast, cited answers — the end-to-end flow behind every customer interaction.

What Is This Feature?

At the heart of every interaction with your AI assistant is the responder — the engine that takes a user's question, finds the most relevant information from your knowledge base, and generates a clear, accurate, cited answer. This deep dive explains how that process works end to end, what makes it fast and reliable, and what safeguards are in place to ensure quality and safety. Understanding this feature helps you appreciate the technical foundation behind every conversation your customers have.

Why It Matters to Your Business

The quality of your AI assistant is judged conversation by conversation. Users form opinions fast — if responses are slow, vague, or factually wrong, they stop trusting the assistant. If they're fast, accurate, and well-sourced, the assistant becomes indispensable.

  • Response quality is your product. Every improvement to the responder — better citation accuracy, lower latency, more relevant retrieval — directly improves what customers experience.
  • Traceability means you can improve. When a response is wrong or slow, you need to know why. Every request is tracked end to end, so your team can diagnose exactly where something went wrong.
  • Safety is built in. Responses go through safety checks before being returned to users. If a response triggers a concern, it's handled gracefully — the user gets an appropriate fallback, not a confusing or harmful output.
  • Resilience against outages. If the primary AI provider has an issue, the system automatically falls back to an alternative — minimizing disruption for your customers.

How a Conversation Works (No Technical Jargon)

When a user sends a message, here's what happens — typically in under a second:

1. The message arrives. The system validates that the user is who they say they are and that they're allowed to use this assistant.

2. Context is assembled. The system looks at the recent conversation history to understand the full context of the question — not just the latest message in isolation.

3. Relevant knowledge is retrieved. The assistant searches your knowledge base for the most relevant content. This is how it can give specific, accurate answers rather than generic responses. It returns the top matching pieces, ranked by relevance.

4. The prompt is built. The retrieved knowledge, conversation history, and any assistant-specific instructions are combined into a carefully structured prompt that gets sent to the AI model.

5. The AI generates a response. The model produces an answer. If it takes too long or returns an error, the system automatically retries or falls back to an alternative model.

6. Safety checks run. The response is checked against safety filters before being returned.

7. Citations are attached. If the answer draws on specific documents in your knowledge base, those sources are attached to the response — so users know where the information came from and can verify it.

8. The response is returned. The user sees the answer, with sources, typically in under a second.

What Good Looks Like

A well-functioning responder produces responses that:

  • Cite their sources. Every factual claim is backed by a link to the specific document it came from.
  • Are fast. Target response time is under 800ms for most queries.
  • Degrade gracefully. If the AI provider is unavailable, users see a clear "please try again" message rather than a broken experience.
  • Are safe. Potentially harmful or inappropriate content is filtered out automatically.

For Your Operations Team

Every request generates a trace — a complete, timestamped record of every step: what was retrieved, what was sent to the AI, what the AI returned, what safety checks ran, and what the final response was. These traces are retained for 7 days and are searchable. When a customer reports a bad response, your team can find the exact trace and understand precisely what happened.

Key metrics your team can monitor:
- Response time (target: under 800ms at the 95th percentile)
- Citation accuracy (target: over 85% of evaluated queries include correct citations)
- Error rate (how often requests fail and why)
- Safety filter activity (how often content is flagged and what type)

What to Expect on the Roadmap

The team is building toward:

1. Full end-to-end tracing for every request (estimated 2 weeks)
2. Safety filter evaluation mode — where filters run but don't block yet, allowing calibration (estimated 3 weeks)
3. A fully tested fallback responder for provider outages (estimated 4 weeks)

These improvements make the responder more transparent, more reliable, and easier to operate — giving your team the tools to maintain and continuously improve conversation quality.