Chain-of-Thought Monitorability
Definition
Chain-of-thought monitorability is the degree to which a monitor can infer a behavior-relevant property of an agent from the agent’s reasoning trace. It is weaker than full faithfulness: the trace need not expose every internal computation if it contains enough legible information for the monitor to detect the relevant property.
Why It Matters
CoT monitoring is a candidate safety layer for frontier reasoning models because it may reveal reward hacking, sandbagging, or other misbehavior before the final answer or action fully hides it. The concept also clarifies why optimizing directly against a chain of thought can be dangerous: strong pressure can make the trace less informative or less legible.
Formalism / Key Objects
- Monitor input: a latent or trace
Z, such as a chain of thought. - Target property:
Y, such as whether behavior is sandbagging, exploiting a tool, or satisfying a process criterion. - Intervention setting: compare control/intervention arms
X=0andX=1. - Instance-level effect:
- Reporting should separate genuine positive-effect instances from finite-sample noise; see Monitorability Metrics and Cross-Fit Filtering.
Connections
- Extends AI Evaluation and Benchmarking beyond answer accuracy into reasoning-trace oversight.
- Related to Calibration because monitorability estimates need uncertainty-aware filtering and reporting thresholds.
- Adjacent to Benchmark Gaming because monitors should detect behavior that exploits the measurement process.
- Part of Explainable AI, but it asks an operational monitoring question rather than requiring complete mechanistic faithfulness.
Common Confusions
- Monitorability is not the same as faithfulness. A trace can be incomplete but still monitorable.
- Monitorability is not just transparency. The monitor must be able to use the available information.
- A low monitorability score can reflect missing information in the trace, monitor weakness, or a noisy evaluation instance.