Campaign configuration reference ​
The campaign: key of driftbench.yaml, or the file named by --config, holds campaign settings. Command-line flags override the file. See the campaign key for how campaign run, campaign plan and ci use these settings. The page lists every field, generated from the zod schemas the CLI uses.
- Keys are snake_case. An unknown key is an error.
- Descriptions come from the schemas. An empty cell means the schema has no description for that field yet.
Contents ​
CampaignConfig ​
The settings of a campaign, each the flag of campaign run it names. A flag on the command line wins over the file.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
specs | list of strings, each not empty, at least 1 item | no | Spec files or folders, read from the working folder. Used by campaign run when the command line names none. | |
dataset | string, not empty | no | A dataset version of the store to run, name or name@version: --dataset. | |
prompts | list of strings, each not empty | no | Prompt versions of the store to record, each name or name@version: --prompt, once per item. | |
targets | list of strings, each not empty | no | The targets every spec runs on, each adapter or adapter:agent: --target, once per item. | |
routes | list of strings, each not empty | no | Targets by spec tag, each tag=target: --route, once per item. | |
modes | list of one of T, F, P, at least 1 item | no | The run modes, such as [T, F]: --mode. | |
repeats | integer, at least 1 | no | Runs per spec, target and mode: --repeats. | |
agent_version | string, not empty | no | The version of the agent under test: --agent-version. | |
store | string, not empty | no | The store file the campaign is kept in, read from the working folder: --store. | |
cache | boolean | no | False neither replays runs from the cache nor writes to it: --no-cache. campaign run only. | |
cache_suite | string, not empty | no | The cache suite of the campaign: --cache-suite. | |
concurrency | integer, at least 1 | no | Cells that run at the same time: --concurrency. | |
budget_usd | number, at least 0 | no | A hard stop on the campaign's spend: --budget-usd. | |
process_command | string, not empty | no | The command that runs the agent of a process target: --process-command. | |
driver | scripted or simulated | no | Who writes the user turns, the spec's own script or a persona: --driver. | |
personas | list of strings, each not empty | no | With the simulated driver, persona files: --persona, once per item. | |
recordings | list of strings, each not empty | no | With the simulated driver, recordings to replay: --recording, once per item. | |
model | boolean | no | With the simulated driver, true speaks through the model port, a paid call: --model. | |
agent_family | string, not empty | no | The model family of the agent under test: --agent-family. | |
judge_families | list of strings, each not empty | no | The model families of the judges: --judge-family, once per item. | |
retry | CampaignRetryConfig | no | How model calls are retried, over the retry settings of the spec target. |
CampaignRetryConfig ​
How model calls are retried, over the retry settings of the spec target.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
max_attempts | integer, at least 1 | no | Attempts per model call, the first included: --retry-attempts. | |
base_delay_ms | integer, at least 0 | no | The wait before the first retry, doubled for each later one: --retry-base-ms. | |
max_delay_ms | integer, at least 0 | no | The longest wait between two attempts: --retry-max-ms. |