Skip to content

JudgeBench: driftbench against every peer ​

Model: claude-haiku-5-5 for every contestant. Unit: pair (official pair-net). driftbench and promptfoo rows are the recorded ledger cells; peer rows are new runs on the same sealed items. Rows marked "pointwise grader, higher score wins" score each reply alone with that tool's pointwise grader; the tool has no pairwise grader.

ContestantGraderScoredriftbench minus thisdriftbench wins / lossesExact pHolm pMissingGrader errorsUSD
driftbenchrecorded cell88.89% (240/270)0
promptfoorecorded cell82.96% (224/270)+5.93 pts27 / 110.01390.1250
deepevalArenaGEval (DeepEval 4.2.8)86.30% (233/270)+2.59 pts15 / 80.211000.4575
inspectmodel_graded_qa (inspect_ai 0.3.278), pointwise grader, higher score wins80.00% (216/270)+8.89 pts31 / 70.0001160.00128000.5264
langfuseAnswer Critic (Langfuse 64ba74f9), pointwise grader, higher score wins74.81% (202/270)+14.07 pts47 / 92.57e-074.37e-06000.3638
langwatchComparison, swap and reconcile (LangWatch e683dd9e)83.70% (226/270)+5.19 pts23 / 90.02010.16020.4228

MIT. driftbench is made by Nexus.