One real evaluation run, documented end to end with the numbers we actually got. Below it, the engagement blueprints behind every service, so you know exactly what you would be buying before you buy it.
A video team needed every clip in a release scored on prompt adherence, color saturation, and motion ghosting, inside a four-hour window. Reviewing hundreds of clips by hand was not scaling. So we built VideoEval and ran it on a real 10-clip batch.
What we did. Automated scoring on every clip: a vision model graded prompt adherence, frame analysis measured saturation and ghosting, and anything uncertain went to a human review queue. Every clip got per-dimension scores, a pass/fail verdict, and the batch got a version-over-version comparison.
The honest finding. A fixed saturation threshold fired on warm color grades and vivid subjects, so absolute saturation pass/fail needs client-specific calibration before it ships. We published that limitation instead of hiding it, and the pipeline still finished the batch in minutes, not hours.
No two engagements are identical, but the shape is consistent. Each blueprint below is a service we actually deliver: what it covers, what you get, and the typical scope. Pick the one that matches your problem.
What it covers: your model's outputs, graded against your rubric by trained reviewers.
Typical scope: 50 to 500 outputs, one to two weeks.
What it covers: a calibrated judge model scoring your outputs at scale, with humans on the uncertain ones.
Typical scope: calibration on 100 to 300 samples, then unlimited scoring runs.
What it covers: structured adversarial testing to find jailbreaks, leaks, and harmful outputs before your users do.
Typical scope: two-week sprint, findings graded P0 to P3.
What it covers: verifying your answers actually come from the retrieved documents, claim by claim.
Typical scope: 100 to 300 Q&A pairs with retrieved documents.
What it covers: ranked human preferences to train your reward models. We deliver the data, never the training.
Typical scope: 500 to 5,000 preference pairs per batch.
What it covers: review of agent traces to catch bad tool calls, loops, and unsafe actions.
Typical scope: 50 to 200 traces across your main workflows.
Yes. It is a real 10-clip evaluation run we conducted and documented, with the actual numbers: 3 passed, 7 failed, 7 flagged for human review, 8.7 seconds per clip. The full report and a 60-second video of the run are public on our sample report page.
We are early, and our first pilot engagements are in progress. This page starts with our own documented evaluation run and the blueprints of what each engagement delivers. As pilots complete, their documented results land here.
Only with written permission. Most teams prefer we do not, so expect anonymized results: the methodology, the scope, and the numbers, without the name.
Yes. Our sample report page shows the format, the grading, and the video of a real run. That is the fastest way to judge our work before a pilot.
Send samples. Get a graded report. That is the whole first step.
A 60-second tour of VideoEval scoring 10 AI-generated clips: prompt adherence, color saturation, motion ghosting, pass/fail verdicts, and the human review queue. Real demo, real numbers.
JudgeMyAI