Open Sourcing Monitorability Evaluations

Summary

This partial ingest is based on an OpenAI Alignment Research Blog post announcing a public subset of the chain-of-thought monitorability evaluation suite, reference code for the g-mean^2 monitorability metric, and a cross-fit filtering strategy for intervention-style estimates. The release frames Chain-of-Thought Monitorability as a practical safety layer: monitors should be able to infer behavior-relevant properties from reasoning traces when those traces remain informative and legible.

Key Claims

• Chain-of-thought monitoring can help catch reward hacking, sandbagging, and other misbehavior by overseeing the model’s reasoning trace rather than only final actions or answers.
• The useful target is monitorability rather than full faithfulness: a trace can omit some intermediate computation yet still contain enough legible information for a monitor to detect the relevant property.
• Open-sourced datasets and reference code should make monitorability easier to track over time and easier for other frontier-model developers to report.
• Intervention-style monitorability metrics are vulnerable to noise-driven positive effects unless eligibility filtering is separated from metric estimation.
• Cross-fit filtering reduces post-selection bias by choosing eligible problem instances on one sample fold and estimating the metric on a disjoint fold.

Methods / Formalism

• The release groups evaluations into intervention evals, process evals, and outcome-property evals.
• For intervention evals, each problem instance compares a control arm X = 0 and intervention arm X = 1 for a binary outcome Y.
• The instance-level total effect is estimated as

$$\widehat{TE}={\hat{p}}_1-{\hat{p}}_0,{\hat{p}}_x=Pr(Y=1\mid X=x).$$

• A naive rule keeps instances with positive observed total effect, but this can retain null-effect instances that looked positive because of finite-sample noise.
• The updated filter uses a one-sided Wald eligibility rule on a selection fold,

$${\widehat{TE}}_1>k{\widehat{SE}}_1,k=1.96,$$

then computes per-instance g-mean^2 on a disjoint estimation fold if the held-out estimate still has positive total effect.

• See Monitorability Metrics and Cross-Fit Filtering for the reusable metric/filtering schema.

Evidence / Experiments

• The blog reports GPT-5.4 thinking results over the open-sourced subset of intervention, process, and outcome-property evals using a CoT-only monitor scope.
• Missing plotted datapoints indicate reporting filters, such as fewer than 10% valid positive-effect intervention instances or too few positive examples for non-intervention evals.
• A synthetic null/mixed-effect demonstration illustrates how naive positive-effect filtering retains false positives, while cross-fit Wald filtering avoids reusing the same positive noise fluctuation for both selection and estimation.
• In the Sandbagging evaluation, cross-fit filtering reinterprets an apparent late-training monitorability drop as a reliability failure caused by too few genuinely positive-effect instances.

Connections

• Extends AI Evaluation and Benchmarking with an evaluation target for reasoning-trace oversight rather than only answer accuracy.
• Connects to Calibration because monitorability reporting depends on finite-sample uncertainty, filtering thresholds, and reliability criteria.
• Complements Benchmark Gaming by measuring whether monitors can detect unwanted behavior before a final score or action hides the issue.
• Relates to Explainable AI through the distinction between faithfulness, legibility, and operational monitoring.

Open Questions

• How stable is CoT monitorability under training methods that optimize directly against monitors?
• Which monitor scopes, such as CoT-only versus CoT-plus-actions, best predict real deployment oversight value?
• How should monitorability metrics be reported when useful interventions produce small or sparse total effects?
• Does cross-fit filtering become too conservative for rare but safety-critical misbehavior?

Citation

Guan, M. Y., Wang, M., Carroll, M., Dou, Z., Wei, A. Y., Williams, M., Arnav, B., Huizinga, J., Kivlichan, I., Glaese, M., Pachocki, J., and Baker, B. (2026). Open Sourcing Monitorability Evaluations. OpenAI Alignment Research Blog.