Skip to content

RubricBench: driftbench against every peer ​

Model: claude-haiku-5-5 for every contestant. Unit: game. driftbench and promptfoo rows are the recorded ledger cells; peer rows are new runs on the same sealed items. Rows marked "pointwise grader, higher score wins" score each reply alone with that tool's pointwise grader; the tool has no pairwise grader.

Note: a few items stay undecided after three passes because the model refused them (an acceptable-use refusal on biology content). They count as wrong for the contestant, as the recorded driftbench cell does.

ContestantGraderScoredriftbench minus thisdriftbench wins / lossesExact pHolm pMissingGrader errorsUSD
driftbenchrecorded cell82.66% (1306/1580)0
promptfoorecorded cell75.57% (1194/1580)+7.09 pts221 / 1096.88e-101.24e-080
deepevalArenaGEval (DeepEval 4.2.8)77.59% (1226/1580)+5.06 pts191 / 1114.85e-066.79e-05101.3583
inspectmodel_graded_qa (inspect_ai 0.3.278), pointwise grader, higher score wins26.58% (420/1580)+56.08 pts898 / 121.47e-2473.66e-246001.3377
langfuseAnswer Critic (Langfuse 64ba74f9), pointwise grader, higher score wins25.06% (396/1580)+57.59 pts930 / 202.59e-2456.21e-2441100.8805
langwatchComparison, swap and reconcile (LangWatch e683dd9e)74.94% (1184/1580)+7.72 pts188 / 669.45e-151.8e-13041.0240

MIT. driftbench is made by Nexus.