Skip to content

Estate files reference ​

The files of an estate test suite: the map driftbench estate scan writes and a person approves, the plan estate plan writes, and the phrase bank and session index that sit beside the map for estate generate. See estate. The page lists every field, generated from the zod schemas the CLI uses.

  • Keys are as written below, some in camelCase. An unknown key is an error.
  • Descriptions come from the schemas. An empty cell means the schema has no description for that field yet.

Contents ​

EstateMap ​

FieldTypeRequiredDefaultDescription
version1yesMap format version, always 1.
runtimestring, not emptyyesThe adapter that read the estate: claude-code, nexus, ...
rootstring, not emptyyesThe estate root, as given; never an absolute private path in a committed file
readAtstring, not emptyyesISO time the scan read the estate.
documentslist of DocumentInfonoDocument authority and non-normative line spans; absent in older maps.
atomslist of AtomyesEvery atom the scan read from the estate.
edgeslist of EdgeyesThe edges of the estate graph, decided by code.
spacestringnoPath of the exported space map of the situation axes
ignorelist of strings, each not emptynoGlobs the scan skipped beside the runtime defaults; lint reads the estate with them again
userSkillstruenotrue when the scan read a user-level skill layer; lint then needs the folder to read it again
followedLinkslist of strings, each not emptynoLinks, relative to the estate root, that the scan read through to a real path outside the root
producedByEstateMap producedBynoWho proposed the model atoms; code atoms say producedBy: code on the atom itself
hashstring, not emptyyesSHA-256 over the sorted atom hashes and edges

DocumentInfo ​

FieldTypeRequiredDefaultDescription
pathstring, not emptyyesPath of the document, relative to the estate root.
roleDocumentRoleyesThe authority of the document as a whole, from its folder, headings and entries.
reasonslist of strings, each not empty, at least 1 itemyesWhat the classification read: the folder, the heading or the entries that gave the role.
sectionslist of DocumentInfo sectionsnoSpans of a live document that carry a non-normative role, such as an example or a history.

DocumentRole ​

ValueMeaning
normativeLive content: its rules, gates and references count.
changelogA record of releases: it holds no rule and gives no test, and it documents the files it names.
historyA dated record of what happened: no rule, no test, and the files it names are documented.
templateA form to fill in: read when a live document sends the reader to it, and it gives no test of its own.
exampleAn illustration, or a counter-example shown so that it is not followed: no rule and no test.
journalA diary or work log of dated entries: no rule and no test.
fixtureData a test or a skill ships: no rule and no test.
archiveMaterial kept for the record, loaded by nothing: no rule and no test.

DocumentInfo sections ​

FieldTypeRequiredDefaultDescription
startLineinteger, at least 1yesFirst line of the section, counted from 1.
endLineinteger, at least 1yesLast line of the section, counted from 1.
roleDocumentRoleyesThe role of this section, which differs from the role of the document.

Atom ​

FieldTypeRequiredDefaultDescription
idstring, not emptyyesStable id: kind, file and a slug of the quote
kindAtomKindyesThe kind of place the atom comes from, which sets its mandatory test types.
titlestring, not emptyyesOne line in plain words
provenanceProvenanceyesWhere the atom is read from: file, line span, quote and hash.
producedBycode or modelyesCode when read from file structure and approved as read; model when split from prose by a model.
roleaxis or detailnoaxisAxis puts the atom in the space map as a target; detail keeps it for the generator only.
approvedbooleannofalseWhether the atom is approved; only approved atoms enter the space map and the plan.
situationstringnoWhen the atom applies, in words; empty means always
evidencelist of EvidencePredicate, at least 1 itemyesWhat a trace shows when the atom is obeyed, as at least one typed predicate.
enforcementlist of stringsno[]Ids of guards or tests that already check it; empty means trusted
fieldsmap of string to any valuenoKind-specific structure read by code: a guard event, a tool schema, a brief field list
replyActAtom replyActnoThe act of the agent reply that the atom governs, when it governs one.

AtomKind ​

ValueMeaning
instructionA rule of the estate: an obligation or a prohibition with its situation, split from prose by a model.
gateAn order rule: before one act, do another, such as reading a file before editing it.
routeA trigger: an ask of one class loads a named skill or helper.
skillA skill unit with its trigger, instructions, references and scripts.
referenceA file a skill or helper names, read at the moment its body names.
scriptAn executable a skill ships, with its inputs and outputs.
subagentA helper contract: name, description, allowed tools, model, brief fields, output shape and acts it must not do.
guardA guard the settings or a skill declare: its event, matcher and command, with its check registry row.
toolA permission rule of the settings that allows, denies or asks for a tool, or a tool a skill declares.
settingA run setting such as the model, the permission mode or an environment key, never its secret value.
stageA step of the step list the estate defines, such as plan, do, verify or report.
registerAn output rule: language, length, format, forbidden words or tone.
hazardA passage of the estate that names a runtime hazard, such as a compaction or a new session.
invariantA static property of the estate files, such as the size ceiling of the standing instructions.

Provenance ​

FieldTypeRequiredDefaultDescription
filestring, not emptyyesPath relative to the estate root
startLineinteger, at least 1yesFirst line of the quote in the file, counted from 1.
endLineinteger, at least 1yesLast line of the quote in the file, counted from 1.
quotestring, not emptyyesThe exact text the atom is read from
hashstring, not emptyyesSHA-256 of the quote, the file and the span; a change here is a changed atom

EvidencePredicate ​

Write one of these forms.

  • a mapping with type: skill_loaded, with the fields below
  • a mapping with type: skill_not_loaded, with the fields below
  • a mapping with type: guard, with the fields below
  • a mapping with type: tool_called, with the fields below
  • a mapping with type: tool_not_called, with the fields below
  • a mapping with type: reference_not_read, with the fields below
  • a mapping with type: subagent_not_dispatched, with the fields below
  • a mapping with type: guard_not_fired, with the fields below
  • a mapping with type: stage_not_reached, with the fields below
  • a mapping with type: artifact_not_created, with the fields below
  • a mapping with type: file_not_written, with the fields below
  • a mapping with type: ordered, with the fields below
  • a mapping with type: stage_reached, with the fields below
  • a mapping with type: workspace, with the fields below
  • a mapping with type: artifact, with the fields below
  • a mapping with type: text, with the fields below
  • a mapping with type: static, with the fields below
  • a mapping with type: custom, with the fields below

With type: skill_loaded:

FieldTypeRequiredDefaultDescription
typeskill_loadedyesThis form holds when the named skill loads in the block.
namestring, not emptyyesThe skill name the trace records on its load.
mininteger, at least 1noThe fewest loads that count; defaults to 1.

With type: skill_not_loaded:

FieldTypeRequiredDefaultDescription
typeskill_not_loadedyesThis form holds when the named skill never loads in the block.
namestring, not emptyyesThe skill name that must not load.

With type: guard:

FieldTypeRequiredDefaultDescription
typeguardyesThis form holds when the named guard records a decision in the trace.
namestring, not emptyyesThe guard name its decision records carry, such as its script file name.
verdictstring, not emptynoWithout it, any verdict counts, as in the guards check
mininteger, at least 0noThe fewest decisions that count; defaults to 1.

With type: tool_called:

FieldTypeRequiredDefaultDescription
typetool_calledyesThis form holds when the agent calls the tool.
labelstring, not emptyyesThe tool label as the trace records it, such as Read or Bash.
inputmap of string to any valuenoInput fields one call must match; without it, any call of the tool counts.
mininteger, at least 1noThe fewest matching calls that count; defaults to 1.

With type: tool_not_called:

FieldTypeRequiredDefaultDescription
typetool_not_calledyesThis form holds when the agent never calls the tool.
labelstring, not emptyyesThe tool label that must not appear in the trace.
inputmap of string to any valuenoOnly forbid calls matching these input assertions; without it, forbid every call.

With type: reference_not_read:

FieldTypeRequiredDefaultDescription
typereference_not_readyesNo successful read of this path or glob in the block.
pathstring, not emptyyesThe path or glob of the reference that must not be read.

With type: subagent_not_dispatched:

FieldTypeRequiredDefaultDescription
typesubagent_not_dispatchedyesNo dispatch of this helper in the block.
namestring, not emptyyesThe name of the helper that must not be dispatched.

With type: guard_not_fired:

FieldTypeRequiredDefaultDescription
typeguard_not_firedyesNo firing of this guard in the block.
namestring, not emptyyesThe name of the guard that must not fire.
verdictstring, not emptynoOnly forbid this verdict; without it, forbid every firing.

With type: stage_not_reached:

FieldTypeRequiredDefaultDescription
typestage_not_reachedyesThis stage never appears in the block.
namestring, not emptyyesThe name of the stage that must not be reached.

With type: artifact_not_created:

FieldTypeRequiredDefaultDescription
typeartifact_not_createdyesThe workspace diff lists no creation of this path or glob.
pathstring, not emptyyesThe path or glob that must not be created.

With type: file_not_written:

FieldTypeRequiredDefaultDescription
typefile_not_writtenyesThe workspace diff lists no creation or change of this path or glob.
pathstring, not emptyyesThe path or glob that must not be created or changed.

With type: ordered:

FieldTypeRequiredDefaultDescription
typeorderedyesThis form holds when the before act comes earlier than the after act.
beforestring, not emptyyesA tool label, a skill name or a stage name
afterstring, not emptyyesA tool label, a skill name or a stage name that must come second.

With type: stage_reached:

FieldTypeRequiredDefaultDescription
typestage_reachedyesThis form holds when the run reaches the named stage.
namestring, not emptyyesThe stage name from the step list of the estate.

With type: workspace:

FieldTypeRequiredDefaultDescription
typeworkspaceyesThis form holds when the workspace changes as listed.
createdlist of stringsnoPaths the run creates in the workspace.
changedlist of stringsnoPaths the run changes in the workspace.
deletedlist of stringsnoPaths the run deletes from the workspace.
othersUntouchedbooleannoWhen true, every path not listed stays untouched.

With type: artifact:

FieldTypeRequiredDefaultDescription
typeartifactyesThis form holds when the run produces a file at the path.
pathstring, not emptyyesWorkspace path of the produced file.
kindstringnoFile kind: xlsx, docx, pptx, pdf, csv, json, text or code; the generator drops any other value.

With type: text:

FieldTypeRequiredDefaultDescription
typetextyesThis form holds when the reply text meets its word and language rules.
mustlist of stringsnoLiteral words the reply must contain.
mustNotlist of stringsnoLiteral words the reply must not contain.
langstringnoLanguage tag the reply is written in, such as EN.

With type: static:

FieldTypeRequiredDefaultDescription
typestaticyesThis form holds when a property of the estate files holds, with no run.
checkstring, not emptyyesA property of the estate files, no run

With type: custom:

FieldTypeRequiredDefaultDescription
typecustomyesThis form names a check that decides the evidence, run by code or judged.
namestring, not emptyyesThe check name, such as ordered, max_words or grounded_values.
optionsmap of string to any valuenoOptions passed to the named check, such as a word limit.

Atom replyAct ​

The act of the agent reply that the atom governs, when it governs one.

FieldTypeRequiredDefaultDescription
idstring, not emptyyesThe act of the reply the rule governs: answer, clarify, refuse, ...
evidencemap of string to any valueyesWhat in the reply shows the act, typed.

Edge ​

FieldTypeRequiredDefaultDescription
fromstring, not emptyyesId of the atom the edge starts at.
tostring, not emptyyesId of the atom the edge ends at, or an event node on an event edge.
kindEdgeKindyesThe relation the edge states, such as route, reference or dispatch.

EdgeKind ​

ValueMeaning
routeA route atom to the skill or helper it loads.
referenceA skill or helper to a file its body names.
scriptA skill to a script it ships.
dispatchA skill or helper to a helper its body names.
enforcesA guard to a rule it checks.
eventA guard to the event node it runs on.
allowsA skill or helper to a tool or skill it may use.
nextA stage to the stage that follows it.
citesA rule to a rule its words name by label.

EstateMap producedBy ​

Who proposed the model atoms; code atoms say producedBy: code on the atom itself

FieldTypeRequiredDefaultDescription
familystring, not emptyyesThe model family that split the prose, code when no model ran
modelstring, not emptyyesThe model that split the prose, or unknown when the model port names none.
atstring matching ^(?:(?:\d\d[2468][048]|\d\d[13579][26]|\d\d0[48]|[02468][048]00|[13579][26]00)-02-29|\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\d|30)|(?:02)-(?:0[1-9]|1\d|2[0-8])))T(?:(?:[01]\d|2[0-3]):[0-5]\d:[0-5]\d(?:\.\d+)?(?:Z))$yesISO time of the scan, the same as readAt.

Plan ​

FieldTypeRequiredDefaultDescription
version1yesPlan format version, always 1.
mapHashstring, not emptyyesHash of the approved map the plan was derived from.
spaceHashstringnoContent hash of the space map the plan used.
rowslist of PlanRowyesThe rows from the contract, the journeys and the pairwise cells.
summaryPlan summaryyesCounts and lists that describe the plan.

PlanRow ​

FieldTypeRequiredDefaultDescription
idstring, not emptyyesRow id: test type and atom id for a contract row, journey.N or pairwise.N otherwise.
atomIdslist of strings, each not empty, at least 1 itemyesAtoms the row exercises: the atom under test, then its skill or helper; a journey lists its path in order.
cellmap of string to stringyesDimension id to value id on the situation axes
testTypeTestTypeyesThe test type the row asks for.
modeRunModeyesHow the row runs: replay, unit or live, set by its test type.
statusone of proposed, approved, strucknoproposedProposed until a person approves or strikes the row; generate skips struck rows.
weightnumber, at least 0no1The share of the run budget the row gets, edited by hand.
notestringnoWhat the row checks, in plain words.

TestType ​

ValueMeaning
obligationThe required act happens, or a forbidden act does not, in the situation of the rule.
outside_triggerNegative control: outside its situation the act does not happen, or an allowed act is not held back.
twinThe obligation again, with the ask put in another phrasing.
ordered_pairThe first act of a gate comes before the second.
skip_pressureThe owner pushes to go faster, and the order of the gate still holds.
route_positiveA positive ask loads the route target.
route_near_missNegative control: an ask close to the trigger that belongs elsewhere does not load the target.
misrouteThe ask names the act without the skill; the right target loads, or the agent says why not.
read_whenThe reference file is read at the moment named.
script_fixtureThe script runs on a fixture, checked by its exit code and output.
dispatchThe helper dispatch carries every brief field the helper needs.
contract_aloneThe helper runs alone: its typed output, and no act it must not do.
stop_resumeThe helper stops mid-task and is resumed with its work intact.
guard_positiveA recorded event gives the guard verdict.
guard_negativeNegative control: a look-alike event leaves the guard silent.
guard_hostileMalformed input gives no crash and a safe verdict.
guard_speedThe guard answers within its time budget.
guard_registerThe guard message follows the output rules.
tool_boundaryTool calls stay inside the schema and the permission rule.
tool_deniedA call the settings deny is never attempted.
tool_deferredA deferred tool is loaded before its first use.
stage_orderThe stage is reached in its order in the step list.
stage_skipThe owner pushes to go faster, and the stage is not skipped.
stage_anytimeA step allowed at any point shows once in the block, wherever, and is not required.
register_sampleA sample of replies to one ask class, judged and counted against the output rule.
hazard_recoveryThe hazard during a long task: the agent recovers, edits nothing out of scope and loses no row.
invariantA static test of the estate files, with no run.
settingA static test that the settings still hold the key, and its value when kept.
journeyA path of atoms from the standing instructions over the estate graph, one block per atom.

RunMode ​

ValueMeaning
replayThe row runs over a recorded session, offline, with no model call.
unitThe row runs offline on one piece of the estate, such as a guard, a script or a helper alone.
liveThe row runs the agent headless in a sandboxed copy of the estate.

Plan summary ​

Counts and lists that describe the plan.

FieldTypeRequiredDefaultDescription
cellsinteger, at least 0yesDistinct pairs of the atoms of a row and its situation values.
coveringArraySizeinteger, at least 0yesRows of the pairwise covering array.
perKindmap of string to integer, at least 0yesContract rows per atom kind, a row counted once for each kind it names.
perTestTypemap of string to integer, at least 0yesRows per test type across contract, journey and pairwise rows.
unenforcedAtomslist of stringsyesIds of approved instruction, gate and register atoms that no guard enforces.
cellsWithoutPhraseinteger, at least 0yesCells with no approved phrase; the plan reads no phrase bank, so it equals cells.
edgesWithoutJourneyinteger, at least 0yesJourney edges that no journey takes.

PhraseBank ​

FieldTypeRequiredDefaultDescription
version1yesThe version of the phrase bank format, 1.
estatestring, not emptyyesThe name of the estate the bank is written for.
mapHashstring, not emptyyesThe hash of the approved map the bank was written against
workspaceslist of WorkspaceSeedno[]The workspace seeds a situation can start from.
situationslist of Situationno[]May be empty: a model-free run still gets the gap report, each row naming the phrase it needs
deviationslist of PhraseBank deviationsno[]The phrases for each deviation class a plan cell can carry.

WorkspaceSeed ​

FieldTypeRequiredDefaultDescription
idstring matching ^[a-z0-9][a-z0-9._-]*$yesA short id, unique among the seeds; a situation names it under workspace.
titlestring, not emptyyesWhat the workspace holds, in a few words.
filesmap of string to stringyesThe files of the workspace, as path and content; a spec inlines them as world.workspace.files.

Situation ​

FieldTypeRequiredDefaultDescription
idstring matching ^[a-z0-9][a-z0-9._-]*$yesA short id, unique among the situations of the bank.
titlestring, not emptyyesWhat the situation is, in a few words.
useSituationUseyesWhat the situation is for, such as setting the atom off or tempting the agent to skip a gate.
atomslist of strings, each not empty, at least 1 itemyesThe atoms this situation exercises
cellmap of string to stringnoSituation axis values the phrases express; a plan row must agree with each of them
workspacestring matching ^[a-z0-9][a-z0-9._-]*$noThe workspace seed the ask starts from
touchesTouchesnoThe workspace paths the ask may touch; every other path stays untouched
provokesSituation provokesnoA call the stand-in agent makes while the situation is in force, such as the removal a PostToolUse guard watches
phraseslist of BankPhrase, at least 1 itemyesThe asks that put the agent in the situation, each approved or not.

SituationUse ​

ValueMeaning
triggerThe situation sets the atom off.
outsideThe situation sits just outside the atom's trigger, so the act must not happen.
near_missThe situation reads like a route's asks but is not one.
misrouteThe situation names the wrong primitive for the work.
pressureThe situation tempts the agent to skip a gate or a stage.
resumeThe situation picks a stopped helper up again.
forcedThe situation instructs the guarded act itself, so a live guard test never waits for the agent to choose it.

Touches ​

The workspace paths the ask may touch; every other path stays untouched

FieldTypeRequiredDefaultDescription
createdlist of strings, each not emptyno[]The workspace paths the ask may create.
changedlist of strings, each not emptyno[]The workspace paths the ask may change.
deletedlist of strings, each not emptyno[]The workspace paths the ask may delete.

Situation provokes ​

A call the stand-in agent makes while the situation is in force, such as the removal a PostToolUse guard watches

FieldTypeRequiredDefaultDescription
toolstring, not emptyyesThe tool of the call, such as Bash.
commandstring, not emptyyesThe command of the call (fictional text); the call runs it as written.

BankPhrase ​

FieldTypeRequiredDefaultDescription
textstring, not emptyyesThe ask, word for word, as the user writes it.
langstring, 2 to 8 charactersyesThe language code of the phrase, such as en or de.
approvedbooleanyesOnly approved phrases reach a spec

PhraseBank deviations ​

FieldTypeRequiredDefaultDescription
classstring, not emptyyesThe deviation class the phrases inject, as a cell names it under deviation.
phraseslist of BankPhrase, at least 1 itemyesThe messages that inject the deviation before the ask, each approved or not.

SessionIndex ​

FieldTypeRequiredDefaultDescription
version1yesThe version of the session index format, 1.
sessionslist of SessionEntry, at least 1 itemyesThe recorded sessions, one entry per transcript.

SessionEntry ​

FieldTypeRequiredDefaultDescription
namestring, not emptyyesThe transcript's file name without .jsonl
situationstring matching ^[a-z0-9][a-z0-9._-]*$yesThe situation of the bank the session was recorded in.
askstring, not emptyyesThe recorded ask: one approved phrase of the situation
from_blockinteger, at least 0no0The session block a generated spec's first block replays from; 0 is the first block.
violateslist of strings, each not emptyno[]The atoms the session breaks: a spec on any of them is planned to fail
notestringnoA free remark about the session, for a person; the generator does not read it.

MIT. driftbench is made by Nexus.