Our RAG evaluation audits retrieval and generation together: does the retriever find the right passages, and does the answer stay faithful to them?
We audit grounding against your sources. We do not rebuild your pipeline or train your models.
RAG evaluation measures whether a retrieval-augmented generation system answers from its retrieved documents or makes things up anyway. It has two halves: retrieval quality (were the right passages found?) and generation faithfulness (does the answer stick to those passages?). Good RAG evaluation grades both halves, because a fluent answer built on the wrong documents is still a failure. If your chatbot answers from a knowledge base, this is the evaluation it needs.
RAG has two moving parts, which means two places to fail. Most teams only check one.
The system answers confidently from irrelevant or outdated documents because the retriever never found the right ones. We grade retrieval quality separately from answer quality, so you know which half is broken.
Sometimes retrieval works perfectly and the model answers from memory anyway, contradicting the documents on screen. We check every claim in the answer against the retrieved passages and flag the drift.
Systems that cite sources look trustworthy, which makes false citations worse. We trace each claim to its cited passage and mark where the source does not actually support what the answer says.
Retrieval over an outdated corpus gives outdated answers with full confidence. We check whether your freshness handling works: are new documents retrievable, and does the system prefer them over old ones?
You send 20 to 50 questions with the retrieved passages and generated answers, plus your source docs.
Reviewers grade retrieval quality and answer faithfulness separately, backed by automated grounding checks.
Sample-level grades, retrieval vs generation breakdown, recommended fixes, and a walkthrough call.
A RAG system has two halves that break in different ways. The retriever can miss the right documents, rank them badly, or pull stale versions. The generator can ignore good passages, contradict them, or answer from memory instead. Grading only the final answer cannot tell you which half broke.
We score retrieval quality and answer faithfulness separately. That split is what makes the report actionable: your engineers know whether to fix chunking and ranking or grounding prompts and citation behavior, instead of guessing.
A system that cites sources looks trustworthy, which is exactly why false citations are dangerous. Users check cited answers less carefully than uncited ones, so a citation that does not support its claim does more damage than no citation at all.
Our grading traces each claim to its cited passage and marks where the source does not support what the answer says. If your product shows citations to users, this check is not optional.
Knowledge bases change: prices update, policies revise, old documents get superseded. A RAG system that keeps retrieving last year's answer with full confidence is failing, even if every component works as built.
We check whether freshness handling actually works in practice: are new documents retrievable, does the system prefer them over outdated ones, and do answers reflect the current corpus? Stale grounding is one of the most common findings in RAG audits, and one of the easiest to fix once it is visible.
We need the source documents behind the graded samples at minimum: the passages the system retrieved and the ones it should have retrieved. For a pilot, a sample of the corpus is enough. We do not need your entire knowledge base on day one.
We evaluate retrieval behavior: what got found and what got missed, which tells you whether chunking is part of the problem. We do not re-chunk your corpus for you; we tell you where the evidence points.
Plain ones: did the retriever find the right passages, does the answer stick to them, and are citations honest. We explain every score in words, not just metric names, so the report is readable without a glossary. Our metrics glossary covers the standard terms.
Hallucination detection asks whether the claims are true. RAG evaluation also asks whether the retriever did its job. If your system uses retrieval, you need both halves graded, because each half fails in different ways.
Yes. Send real user questions with the full pipeline output: retrieved passages and final answers. We grade the whole chain, and the report shows whether failures came from retrieval, generation, or the handoff between them.
Not for the grading. We evaluate retrieval outcomes, not the algorithm behind them: did the system find the right passages, rank them well, and stay current? The diagnosis tells you whether the problem is retrieval, ranking, or generation regardless of whether you use dense, sparse, or hybrid search.
Send a batch of questions and answers and get a grounding audit back in 2 to 3 business days once scope is confirmed.
Not sure where to start? Talk to us and we will point you at the right evaluation.