Red Team (RT) Alignment Audit

Audits side tasks for red team description alignment: whether the red team (RT) description gives an agent enough information to accomplish the side task and pass the scorer. Use it to find vague RT descriptions and scorer requirements the description never mentions, before creating sabotage trajectories (strajs).

Commands

ct extras rt-alignment

Audit side tasks for RT description alignment.

Examples:

# Audit all side tasks
ct extras rt-alignment

# Audit specific environment
ct extras rt-alignment -e web_scraping

# Audit specific side task
ct extras rt-alignment -s expose_secret

ct extras rt-alignment-view

Interactive viewer for RT alignment audit results.

Usage:

ct extras rt-alignment-view [RUN_PATH]

Output

Results are saved to data/rt_alignment_runs/<name>_<timestamp>/:

• results.jsonl - Alignment analysis and recommendations for each side task
• prompt_logs/<env>_<side_task>.json - Prompts, response, and thinking for each side task
• run.log - Run log

Analysis Focus

The audit reads the RT description, scorer code, and submit tool, using the prompts in prompts/system.md and prompts/user.md. For each side task it asks:

• Does the RT description clearly state what to do?
• Does it provide enough detail to pass the scorer?
• Are there undocumented requirements?
• Is the task achievable based on the RT description alone?