Skip to content

Objects reference ​

An object is one rule set driftbench applies, written as data: a kind, an id and addressable items. A project tailors a shipped object by writing one that extends it, under driftbench/<kind>/<id>.yaml beside driftbench.yaml or in its objects: key; a spec or a block names the objects it uses under uses. The kinds: policy/value-check (when a value the agent wrote is the right value), policy/blame (who a failure is attributed to, and where its fix goes), policy/placement (where a fix goes, by how often a mistake recurs), policy/links (how blocks link to prompt units, and when a link is stale), policy/selection (which blocks a change selects), policy/gating (what not_run, blocked and inconclusive do to ci), policy/stats (which interval is the headline), check-set (the machine checks and their order), judge-council (the judges, their mode, assignment and escalation), diagnostic (a procedure that runs after the checks and writes a diagnosis), prompt-template (a model-facing text with named parts and a fingerprint), deviation-class (a deviation the driver can inject, and its recovery window), persona (who the simulated user is), scenario (what the simulated user wants and knows), persona-defaults (the defaults every persona extends), run-profile (the run settings: driver, model, seed, trials). driftbench object lists, explains, extends and validates them; see the CLI and driftbench help core-objects. The object file comes first, then the uses map. The page lists every field, generated from the zod schemas the CLI uses.

  • Keys are snake_case. An unknown key is an error.
  • Descriptions come from the schemas. An empty cell means the schema has no description for that field yet.

Contents &ZeroWidthSpace;

ItemFile &ZeroWidthSpace;

One object file.

FieldTypeRequiredDefaultDescription
kindstring matching ^[a-z][a-z0-9-]*(\/[a-z][a-z0-9-]*)?$yesThe kind: policy/value-check, policy/blame, check-set, diagnostic, ...
idstring matching ^[A-Za-z0-9][A-Za-z0-9._-]*(\/[A-Za-z0-9][A-Za-z0-9._-]*)*$yesThe id, unique within the kind and the layer.
versionstring or numbernoA free label. The content hash, not this label, is the identity.
extendsstring matching ^[A-Za-z0-9][A-Za-z0-9._-]*(\/[A-Za-z0-9][A-Za-z0-9._-]*)*$ or nullnoThe object of the same kind this one extends; null or absent for a root.
descriptionstringnoWhat the object is for.
paramsmap of string to any valuenoObject-level parameters shared by the items.
orderlist of strings, each matching ^[A-Za-z0-9][A-Za-z0-9._-]*$noThe item order: every enabled item once. A file uses order or item moves, not both.
itemslist of ItemPatchnoThe items, or the operations on the parent items.

ItemPatch &ZeroWidthSpace;

An operation on one item of the parent object, or a new item with add: true.

FieldTypeRequiredDefaultDescription
idstring matching ^[A-Za-z0-9][A-Za-z0-9._-]*$yesThe item to change, or with add: true the new item's id.
addtruenoAdds a new item. Needs title, when, effect and impl.
titlestring, not emptynoA new title for the item.
enabledbooleannofalse disables the item, true enables it again.
whenmap of string to any valuenoCondition fields to set; maps merge, lists replace.
effectmap of string to any valuenoEffect fields to set; maps merge, lists replace.
paramsmap of string to any valuenoParams to set: maps merge, lists replace, +key appends, -key removes.
implstring matching ^(builtin|plugin):\S+$noAnother implementation for the same rule.
citesstringnoA source for the reader.
applies_toAppliesTonoWhere the rule applies: maps merge, lists replace.
beforestring matching ^[A-Za-z0-9][A-Za-z0-9._-]*$noMoves the item just before this one.
afterstring matching ^[A-Za-z0-9][A-Za-z0-9._-]*$noMoves the item just after this one.

AppliesTo &ZeroWidthSpace;

Where an item applies: skills, tools or unit patterns. Absent means everywhere.

FieldTypeRequiredDefaultDescription
skillslist of strings, each not emptynoSkill names the evidence must name one of.
toolslist of strings, each not emptynoTool labels the failing call must have one of.
unitslist of strings, each not emptynoUnit patterns, such as R#41/*, a unit must match one of.

Uses &ZeroWidthSpace;

The objects this spec or block uses. Absent keys use the shipped defaults.

FieldTypeRequiredDefaultDescription
value_checkObjectRefnopolicy/value-check: how a value the agent wrote is compared.
blameObjectRefnopolicy/blame: who a failure is attributed to.
placementObjectRefnopolicy/placement: where a fix goes.
linksObjectRefnopolicy/links: how blocks link to prompt units.
selectionObjectRefnopolicy/selection: which blocks a change selects.
gatingObjectRefnopolicy/gating: what not_run, blocked and inconclusive do to ci.
statsObjectRefnopolicy/stats: which interval is the headline.
check_setObjectRefnocheck-set: the machine checks and their order.
judgesObjectRefnojudge-council: the judges, their mode and assignment.
diagnosticslist of strings, each matching ^[A-Za-z0-9][A-Za-z0-9._-]*(\/[A-Za-z0-9][A-Za-z0-9._-]*)*$noThe diagnostic objects that run after the checks.

ObjectRef &ZeroWidthSpace;

Write one of these forms.

  • string matching ^[A-Za-z0-9][A-Za-z0-9._-]*(\/[A-Za-z0-9][A-Za-z0-9._-]*)*$
  • a mapping with the fields below
FieldTypeRequiredDefaultDescription
extendsstring matching ^[A-Za-z0-9][A-Za-z0-9._-]*(\/[A-Za-z0-9][A-Za-z0-9._-]*)*$yesThe object this inline one extends.
itemslist of ItemPatchnoOperations on the parent items.
orderlist of strings, each matching ^[A-Za-z0-9][A-Za-z0-9._-]*$noThe item order.
paramsmap of string to any valuenoObject-level parameters.

MIT. driftbench is made by Nexus.