Skip to content

Personas and red-team reference ​

The files that drive a simulated or adversarial user: a persona and a recording for driftbench simulate and campaign run --driver simulated, a cassette for a simulated user defined in the blocks (simulate --cassette), a red-team pack for perturb and attack, and a red-team plugin. See simulated user, attack and red-team plugins. The page lists every field, generated from the zod schemas the CLI uses.

  • Keys are as written below. An unknown key is an error.
  • Descriptions come from the schemas. An empty cell means the schema has no description for that field yet.

Contents ​

Persona ​

A persona for the simulated user. Fictional.

FieldTypeRequiredDefaultDescription
idstring matching ^[a-z0-9][a-z0-9-]*$yesAn id: lower case letters, digits and hyphens.
namestring, not emptyyesThe name the person goes by. Fictional.
personastring, not emptyyesWho the person is and how they write.
goalstring, not emptyyesWhat the person wants from the conversation.
factsmap of string to string, not emptyyesWhat the person knows and gives only when asked, by name.
reveal_when_askedmap of string to list of strings, each not empty, at least 1 itemyesFor each fact, the patterns of an agent reply that asks for it.
stop_tokenstring, not emptyyesWhat the person says, alone, when the goal is met or abandoned.
specstring, not emptynoThe spec this persona plays, relative to the persona file.

UserRecording ​

Recorded answers of a simulated user, replayed instead of calling a model.

FieldTypeRequiredDefaultDescription
familystring, not emptyyesThe family recorded: who wrote the answers.
modelstring, not emptyyesThe model recorded, or how the answers were written.
personastring, not emptyyesThe persona id the answers belong to.
answerslist of strings, each not empty, at least 1 itemyesThe answers, one per turn, in order.

UserCassette ​

Recorded responses of the simulated user, replayed instead of calling a model.

FieldTypeRequiredDefaultDescription
familystring, not emptyyesThe model family the responses were recorded from.
modelstring, not emptyyesThe model recorded.
entriesmap of string to stringyesEach response, by the SHA-256 of its request (seed, occurrence, system, user and token cap).

PackFile ​

One file of the red-team pack: the phrasings of one deviation class.

FieldTypeRequiredDefaultDescription
classone of D1_off_topic, D2_goal_shift, D3_contradiction, D4_loose_confirmation, D5_language_switch, D6_instruction_in_data, D7_silence_or_burst, D8_tool_failure, D9_out_of_scope, D10_return, D11_pii_leak, D12_prompt_extractionyesThe class every phrasing of the file belongs to.
titlestring, not emptynoA title for a reader; no command reads it.
stageslist of strings, each not empty, at least 1 itemnoThe stage names this file may use in at_stage. Any other name makes the pack invalid.
conversationlist of Phrasing, at least 1 itemyesPhrasings for conversation specs.
tasklist of Phrasing, at least 1 itemnoPhrasings for task specs.

Phrasing ​

One phrasing of a deviation.

FieldTypeRequiredDefaultDescription
idstring matching ^[a-z0-9][a-z0-9-]*$yesAn id, unique in the pack: lower case letters, digits and hyphens.
approvedbooleanyesTrue once a person has read the phrasing and accepts it.
notestringnoWhy the phrasing exists, for the reviewer.
ends_conversationbooleannoIn attack, stop only after this conversation phrasing has checked, passing reply expectations.
userstringnoThe message the user sends. Empty is silence (D7 only).
burstlist of strings, each not empty, at least 2 itemsnoMessages that arrive together (D7 only).
worldPhrasing worldnoA change of the world (D6 on a conversation).
instructionstring, not emptynoThe instruction the world carries (D6).
errorstring, not emptynoThe error the tool returns (D8 on a conversation).
toolstring, not emptynoThe tool that fails (D8 on a conversation).
filestring, not emptynoThe planted file (D6 on a task).
contentstring, not emptynoThe text of the planted file or obstacle (task).
mustlist of string, not empty or Phrasing mustnoPatterns the reply to this phrasing must match (conversation only). Merged into the injected block. A pattern with at_stage applies only at those stages.
must_notlist of string, not empty or Phrasing must_notnoPatterns the reply to this phrasing must not match (conversation only). A pattern with at_stage applies only at those stages.

Phrasing world ​

A change of the world (D6 on a conversation).

FieldTypeRequiredDefaultDescription
toolsmap of string to Phrasing world tools valuenoForced tool results and errors, by tool name.
storemap of string to any valuenoA seeded store.
workspacePhrasing world workspacenoThe files the agent starts from.
fixturesmap of string to any valuenoFree fixtures.

Phrasing world tools value ​

A forced result or error for one tool, from the block where it is set onwards.

FieldTypeRequiredDefaultDescription
resultJSON valuenoThe result the tool returns.
errorstringnoThe error the tool returns.
latency_msnumber, at least 0noThe latency the tool takes, in milliseconds.

Phrasing world workspace ​

The files the agent starts from.

FieldTypeRequiredDefaultDescription
fromstring, not emptynoA folder the workspace starts from. A path written ./<path> is read from the folder of the spec file; any other relative path from the folder the run starts in.
filesmap of string to stringnoFiles the workspace starts with: path to content.
binarymap of string to string, base64noFiles the workspace starts with, byte for byte: path to base64 content. For files that are not text, such as the xlsx a parent block left. A path may not be in both files and binary.
summarystring, not emptynoWhat the workspace holds, in words, so a reader of the spec sees it. spec lint --summaries warns when a folder named by from has none.

Phrasing must &ZeroWidthSpace;

FieldTypeRequiredDefaultDescription
patternstring, not emptyyesThe pattern.
at_stagelist of strings, each not empty, at least 1 itemyesThe stages the pattern applies at, named in the stages of the file. The stage is the one the previous block of the spec expects.

Phrasing must_not &ZeroWidthSpace;

FieldTypeRequiredDefaultDescription
patternstring, not emptyyesThe pattern.
at_stagelist of strings, each not empty, at least 1 itemyesThe stages the pattern applies at, named in the stages of the file. The stage is the one the previous block of the spec expects.

RedteamPlugin &ZeroWidthSpace;

One red-team plugin.

FieldTypeRequiredDefaultDescription
idstring matching ^[a-z0-9][a-z0-9-]*$yesThe plugin id: lower case letters, digits and hyphens. Unique in the pack.
familyone of harm, security, policy, reliabilityyesThe family the plugin belongs to.
titlestring, not emptyyesA short title for reports.
sourcestring, not emptyyesThe upstream plugin this one was adapted from (MIT, see NOTICE).
templatestring, not emptyyesThe generation template: asks for attack requests.
rubricstring, not emptyyesThe grader rubric: judges one reply as pass or fail.

MIT. driftbench is made by Nexus.