Skip to content

LongJudgeBench writing preference: driftbench against every peer ​

Model: claude-haiku-5-5 for every contestant. Unit: game. driftbench and promptfoo rows are the recorded ledger cells; peer rows are new runs on the same sealed items. Rows marked "pointwise grader, higher score wins" score each reply alone with that tool's pointwise grader; the tool has no pairwise grader.

ContestantGraderScoredriftbench minus thisdriftbench wins / lossesExact pHolm pMissingGrader errorsUSD
driftbenchrecorded cell61.27% (212/346)0
promptfoorecorded cell56.94% (197/346)+4.34 pts49 / 340.1240.8670
driftbench-reader-firstrecorded cell67.63% (234/346)-6.36 pts60 / 820.07770
deepevalArenaGEval (DeepEval 4.2.8)58.38% (202/346)+2.89 pts46 / 360.321000.4740
inspectmodel_graded_qa (inspect_ai 0.3.278), pointwise grader, higher score wins11.56% (40/346)+49.71 pts178 / 64.19e-459.63e-44000.3847
langfuseAnswer Critic (Langfuse 64ba74f9), pointwise grader, higher score wins14.45% (50/346)+46.82 pts172 / 102.96e-396.51e-38000.2047
langwatchComparison, swap and reconcile (LangWatch e683dd9e)52.60% (182/346)+8.67 pts42 / 125.21e-050.000677000.3089

MIT. driftbench is made by Nexus.