JudgeMyAIJUDGEMYAI · FREE DOWNLOAD
The LLM Eval Checklist
28 checks to run before you trust an AI answer in production.
The same checklist we run in our own evaluations.
2 pages. Print it, pin it, run it on every release.
What’s inside
Seven sections of pass/fail checks you can run today:
1. Accuracy — claims verified, numbers exact, no invented citations
2. Hallucination — every claim classified: supported, unsupported, contradicted
3. Safety — policy checks, prompt-injection resistance, PII handling
4. Bias & Fairness — both-order judging to kill position bias
5. Instruction Following — format, completeness, constraint respect
6. Completeness & Clarity — nothing cut off, caveats up front, stable across runs
7. Grounding (RAG) — every claim traceable to a retrieved chunk
How to use it
Run every check on each release candidate. Mark pass or fail. Grade every failure P0 (blocks shipping), P1 (fix this week), P2 (fix this month), or P3 (polish). One P0 means do not ship. Fix it, then re-run the full list.
Want this done for you?
Get a free pilot evaluation of your AI system. We run the full checklist, grade every issue P0–P3, and hand you a report you can act on.
Start a free pilotor email us directly: pilot@judgemyai.com