Skip to content

What you can change in an estate ​

This page is the help topic validate-estate-settings (driftbench help validate-estate-settings). Its worked example runs in CI, and each output line it shows is compared with what the commands print.

What it is ​

An estate scan reads your agent's files. You choose how it reads them in three places.

  • Settings: the keys of driftbench.estate.yaml at the root of the estate.
  • Files: what the scan reads beside the estate, and what it writes for you to keep.
  • Ports: where your own code or service plugs in, and the environment variables that choose a model or a key.

A command-line option always wins over its key in the settings file. Every path in the settings file is relative to the estate root.

When to use it ​

  • Before the first scan of a new estate: write the settings file once, so every scan reads the same way.
  • Before you change a label set, a model or a threshold: see which recorded answers the change costs.
  • When you want your own labeler: plug it into the port instead of changing driftbench.

Commands and every option ​

The settings file, driftbench.estate.yaml. Every key is optional.

KeyRead byWhat it holdsExample
ignoreestate scan, estate lintglobs of paths the scan skips, beside the defaultsignore: [logs/, drafts/**]
externalsestate scanfolders outside the estate a path may nameexternals: [docs/]
aliasesestate scanan alias and the folder it stands foraliases: { REF: .claude/skills/x/references }
documentsestate scana glob and the role of the documents it matches; it wins over the recorded label and over structuredocuments: { 'archive/**': archive, 'forms/*.md': template }
scriptsestate unitthe fixtures of each script the unit runner runsscripts: { .claude/guards/check.py: [{ args: [--dry-run], expect: { exitCode: 0 } }] }
labelsestate scan --pointerthe label sets: a file, or the sets inlinelabels: labels.yaml
labelerestate scan --pointerone labeler per class, <class>:<client>, never a model; without it, the default stack of both classeslabeler: [system1:openai-decisions, system2:anthropic]
thresholdestate scan --pointerlabeler, pass_at, fail_at, margin_cut, each from 0 to 1threshold: { pass_at: 0.9, fail_at: 0.1 }
ledgerestate scan --pointer, radius ledgerthe ledger of unit idsledger: driftbench.ledger.json

The options that win over a key: --labels <file>, --labeler <class>:<client>[:<model>], --labeler-threshold <p>, --labeler-pass-at <p>, --labeler-fail-at <p>, --labeler-margin-cut <m> and --ledger <file> of estate scan, and --ledger <file> of radius ledger. A path given as an option is relative to the folder you run the command from. --update writes the ledger; without it the scan only reads it.

The files.

FileWho writes itWhat a change doesExample
driftbench.estate.yamlyouthe settings aboveledger: driftbench.ledger.json
the label sets fileyouwhich labels a unit can take; a set's description is in the request, so a new description re-asks that set onlylabels: [{ set: house, description: How replies are worded., labels: [{ name: rule, description: A unit that fixes a word. }] }]
the ledgerestate scan --pointer --updatethe unit ids of the last scan; commit it, so radius ledger reads an edit against itdriftbench.ledger.json
the cassetteestate scan --pointer --recordthe recorded answers; a scan without --record needs no key and no network--cassette estate/cassette.jsonl
guard-state.yamlyou, beside a mapthe enforcement mode of each guard, read by estate diffversion: 1 and guards: [{ guard: guard.settings.x, mode: warn, since: 2026-10-01 }]
a change manifestradius ledger, estate diff --change-out, or youwhat changed, read by radius and campaign run --changechanges: [{ kind: prompt, id: instruction.standing.r7 }]

The ports.

PortHow it plugs inExample
a labeler endpoint<class>:http:<url>; the bearer token in the variable --labeler-key nameslabeler: system2:http:https://labels.example.com/label
a labeler module<class>:module:<path>; a module that exports id, cls and label(request)labeler: system2:module:labeler.mjs
ANTHROPIC_LABELER_MODELthe model of anthropic; default claude-haiku-5-5ANTHROPIC_LABELER_MODEL=claude-sonnet-5-5
JEV_MODELthe model of the system 1 client of the Nexus platform, the default system 1 labeler (on the Nexus adapter page)JEV_MODEL=<model>
OPENAI_DECISIONS_MODELthe model of openai-decisions; it has no defaultOPENAI_DECISIONS_MODEL=<model>
ANTHROPIC_API_KEY, JEV_API_KEY with JEV_URL or JEV_ROUTE, OPENAI_API_KEYthe keys of a --record run; never printed or recordedANTHROPIC_API_KEY set in the shell

No key of the settings file names a model. A labeler with a model in the file stops the scan and names the variable to use, so a file you share never pins a model.

Inputs &ZeroWidthSpace;

  • The estate folder, with driftbench.estate.yaml at its root when you have one.
  • For --pointer: a cassette, and with --record the keys of the services the labelers call.
  • For radius ledger: the map of the last scan and the ledger it wrote.

What a change costs in recorded answers:

ChangeRecorded answers it voids
a label set's description or labelsthe answers of that set
a model, by its variablethe answers of that labeler
a thresholdnone: the labels are decided again from the cassette
a labeler left out of the stacknone
the ledger pathnone
an edit of the estate's prosenone of the old answers; the windows that hold an edited unit need new ones

Outputs and how to read them &ZeroWidthSpace;

With a settings file, estate scan --pointer prints Pointer settings from driftbench.estate.yaml: <keys>. first: the keys it read. The report then names the labelers (Pointer reader: ...), the units kept from the ledger and labelled by this run, and the requests: answered, sent and recorded. A run that prints 0 sent cost nothing.

A pointer key with the wrong shape stops the scan with exit 2 and names the key. The other keys that have the wrong shape are left out of the scan without a note.

radius ledger prints the count of each edit class, then how many units rerun and how many are advisory only.

Worked example &ZeroWidthSpace;

The fixture estate, copied, with a settings file. It declares the role of each document the runtime does not load as instructions (documents:, which wins over a recorded label), so no document-roles request is made. It selects the fixture's label set settings-labels.yaml, the shipped set without its condition label: this example shows the settings file, and the recorded answers of that set in settings-cassette.jsonl answer every request, so nothing is sent. It sets no labeler, so the scan reads with the default stack of both classes.

sh
cp -R examples/estates/fixture estate
yaml
labels: settings-labels.yaml
threshold: { pass_at: 0.9, fail_at: 0.1 }
ledger: driftbench.ledger.json
documents:
  .claude/skills/invoice-reconcile/references/currency-table.md: fixture
  .claude/skills/invoice-reconcile/references/matching-rules.md: normative
  .claude/skills/invoice-reconcile/references/old-tolerances.md: archive
  .claude/skills/shipment-tracker/data/shipments.txt: history
  .claude/skills/weekly-report/templates/report.md: template
sh
driftbench estate scan estate --pointer --cassette estate/settings-cassette.jsonl --update --out map.yaml
text
Pointer settings from driftbench.estate.yaml: labels, threshold, ledger.
  Requests: 91 answered, 0 sent, 0 recorded.
  Units since the last ledger: new 136.

Edit one rule, then ask what the edit reruns. The ledger the settings file names is the baseline.

sh
perl -pi -e 's/more than two days/more than one day/' estate/CLAUDE.md
sh
driftbench radius ledger map.yaml --out change.yaml
text
Units: 133 unchanged, 2 context, 0 moved, 1 changed, 0 deleted, 0 new.
  1 unit reruns, 2 advisory only; 1 change.
Wrote change.yaml.

Plug in a labeler of your own. This module calls a line that starts with "Never" a prohibition.

js
export default {
  id: 'never-lines',
  cls: 'system2',
  label(request) {
    const labels = request.window.units.map((unit) => ({
      units: String(unit.n),
      label: /^(?:- )?(?:\*\*[^*]+\*\* )?Never\b/.test(unit.text) ? 'prohibition' : 'none',
    }));
    return { labels };
  },
};
yaml
labeler: system2:module:labeler.mjs
ledger: driftbench.ledger.json
sh
driftbench estate scan estate --pointer --cassette estate/module-cassette.jsonl --record --out module-map.yaml
text
Pointer settings from driftbench.estate.yaml: labeler, ledger.
  Pointer reader: system 2 module:never-lines never-lines.
  Requests: 95 answered, 95 sent, 95 recorded.
  Units since the last ledger: new 20, unchanged 135, changed 1.
The ledger estate/driftbench.ledger.json is not written: pass --update to write it.

A model in the settings file is refused:

yaml
labeler: system2:anthropic:claude-haiku-5-5
sh
driftbench estate scan estate --pointer --cassette estate/cassette.jsonl --out refused.yaml # exit 2

Common errors &ZeroWidthSpace;

MessageCauseFix
labeler: "<spec>" names a model; a model is set by environment variable, <VARIABLE> for <client>the settings file names a modelremove the model and set the variable
threshold: <key> is not a thresholda key other than labeler, pass_at, fail_at, margin_cutuse one of the four
name the ledger with --ledger <file>, or with ledger: in driftbench.estate.yamlradius ledger found no ledgeradd ledger: to the settings file, or pass --ledger
the cassette has no answer for request <key>a changed set, model or prose needs an answer the cassette lacksscan with --record, or undo the change

MIT. driftbench is made by Nexus.