Personas and red-team reference ​
The files that drive a simulated or adversarial user: a persona and a recording for driftbench simulate and campaign run --driver simulated, a cassette for a simulated user defined in the blocks (simulate --cassette), a red-team pack for perturb and attack, and a red-team plugin. See simulated user, attack and red-team plugins. The page lists every field, generated from the zod schemas the CLI uses.
- Keys are as written below. An unknown key is an error.
- Descriptions come from the schemas. An empty cell means the schema has no description for that field yet.
Contents ​
- Persona
- UserRecording
- UserCassette
- PackFile
- Phrasing
- Phrasing world
- Phrasing world tools value
- Phrasing world workspace
- Phrasing must
- Phrasing must_not
- RedteamPlugin
Persona ​
A persona for the simulated user. Fictional.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
id | string matching ^[a-z0-9][a-z0-9-]*$ | yes | An id: lower case letters, digits and hyphens. | |
name | string, not empty | yes | The name the person goes by. Fictional. | |
persona | string, not empty | yes | Who the person is and how they write. | |
goal | string, not empty | yes | What the person wants from the conversation. | |
facts | map of string to string, not empty | yes | What the person knows and gives only when asked, by name. | |
reveal_when_asked | map of string to list of strings, each not empty, at least 1 item | yes | For each fact, the patterns of an agent reply that asks for it. | |
stop_token | string, not empty | yes | What the person says, alone, when the goal is met or abandoned. | |
spec | string, not empty | no | The spec this persona plays, relative to the persona file. |
UserRecording ​
Recorded answers of a simulated user, replayed instead of calling a model.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
family | string, not empty | yes | The family recorded: who wrote the answers. | |
model | string, not empty | yes | The model recorded, or how the answers were written. | |
persona | string, not empty | yes | The persona id the answers belong to. | |
answers | list of strings, each not empty, at least 1 item | yes | The answers, one per turn, in order. |
UserCassette ​
Recorded responses of the simulated user, replayed instead of calling a model.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
family | string, not empty | yes | The model family the responses were recorded from. | |
model | string, not empty | yes | The model recorded. | |
entries | map of string to string | yes | Each response, by the SHA-256 of its request (seed, occurrence, system, user and token cap). |
PackFile ​
One file of the red-team pack: the phrasings of one deviation class.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
class | one of D1_off_topic, D2_goal_shift, D3_contradiction, D4_loose_confirmation, D5_language_switch, D6_instruction_in_data, D7_silence_or_burst, D8_tool_failure, D9_out_of_scope, D10_return, D11_pii_leak, D12_prompt_extraction | yes | The class every phrasing of the file belongs to. | |
title | string, not empty | no | A title for a reader; no command reads it. | |
stages | list of strings, each not empty, at least 1 item | no | The stage names this file may use in at_stage. Any other name makes the pack invalid. | |
conversation | list of Phrasing, at least 1 item | yes | Phrasings for conversation specs. | |
task | list of Phrasing, at least 1 item | no | Phrasings for task specs. |
Phrasing ​
One phrasing of a deviation.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
id | string matching ^[a-z0-9][a-z0-9-]*$ | yes | An id, unique in the pack: lower case letters, digits and hyphens. | |
approved | boolean | yes | True once a person has read the phrasing and accepts it. | |
note | string | no | Why the phrasing exists, for the reviewer. | |
ends_conversation | boolean | no | In attack, stop only after this conversation phrasing has checked, passing reply expectations. | |
user | string | no | The message the user sends. Empty is silence (D7 only). | |
burst | list of strings, each not empty, at least 2 items | no | Messages that arrive together (D7 only). | |
world | Phrasing world | no | A change of the world (D6 on a conversation). | |
instruction | string, not empty | no | The instruction the world carries (D6). | |
error | string, not empty | no | The error the tool returns (D8 on a conversation). | |
tool | string, not empty | no | The tool that fails (D8 on a conversation). | |
file | string, not empty | no | The planted file (D6 on a task). | |
content | string, not empty | no | The text of the planted file or obstacle (task). | |
must | list of string, not empty or Phrasing must | no | Patterns the reply to this phrasing must match (conversation only). Merged into the injected block. A pattern with at_stage applies only at those stages. | |
must_not | list of string, not empty or Phrasing must_not | no | Patterns the reply to this phrasing must not match (conversation only). A pattern with at_stage applies only at those stages. |
Phrasing world ​
A change of the world (D6 on a conversation).
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
tools | map of string to Phrasing world tools value | no | Forced tool results and errors, by tool name. | |
store | map of string to any value | no | A seeded store. | |
workspace | Phrasing world workspace | no | The files the agent starts from. | |
fixtures | map of string to any value | no | Free fixtures. |
Phrasing world tools value ​
A forced result or error for one tool, from the block where it is set onwards.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
result | JSON value | no | The result the tool returns. | |
error | string | no | The error the tool returns. | |
latency_ms | number, at least 0 | no | The latency the tool takes, in milliseconds. |
Phrasing world workspace ​
The files the agent starts from.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
from | string, not empty | no | A folder the workspace starts from. A path written ./<path> is read from the folder of the spec file; any other relative path from the folder the run starts in. | |
files | map of string to string | no | Files the workspace starts with: path to content. | |
binary | map of string to string, base64 | no | Files the workspace starts with, byte for byte: path to base64 content. For files that are not text, such as the xlsx a parent block left. A path may not be in both files and binary. | |
summary | string, not empty | no | What the workspace holds, in words, so a reader of the spec sees it. spec lint --summaries warns when a folder named by from has none. |
Phrasing must ​
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
pattern | string, not empty | yes | The pattern. | |
at_stage | list of strings, each not empty, at least 1 item | yes | The stages the pattern applies at, named in the stages of the file. The stage is the one the previous block of the spec expects. |
Phrasing must_not ​
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
pattern | string, not empty | yes | The pattern. | |
at_stage | list of strings, each not empty, at least 1 item | yes | The stages the pattern applies at, named in the stages of the file. The stage is the one the previous block of the spec expects. |
RedteamPlugin ​
One red-team plugin.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
id | string matching ^[a-z0-9][a-z0-9-]*$ | yes | The plugin id: lower case letters, digits and hyphens. Unique in the pack. | |
family | one of harm, security, policy, reliability | yes | The family the plugin belongs to. | |
title | string, not empty | yes | A short title for reports. | |
source | string, not empty | yes | The upstream plugin this one was adapted from (MIT, see NOTICE). | |
template | string, not empty | yes | The generation template: asks for attack requests. | |
rubric | string, not empty | yes | The grader rubric: judges one reply as pass or fail. |