Anthropic Alignment has introduced CHIVE (Counterfactual Hypothesis Investigation Via Edits), an agentic pipeline designed to identify and explain unexpected large language model (LLM) behaviors. This pipeline uses counterfactual prompt edits to investigate these behaviors, generating data that serves two primary purposes: evaluating interpretability tools and training models to predict behavioral changes.
CHIVE operates by taking transcripts from any source, discovering unexpected behaviors in a target model, and then investigating each behavior through counterfactual prompt edits. Each investigation produces an open-ended explanation of the behavior and the measured outcomes of the counterfactual experiments, which serve as evaluation labels.
Key Points
- CHIVE is an agentic pipeline that discovers unexpected LLM behaviors and explains them with counterfactual prompt edits.
- The pipeline generates two types of data: open-ended explanations of behavior and measured outcomes from counterfactual experiments.
- Evaluation using CHIVE data found that activation-reading interpretability tools provided no uplift in predicting counterfactual outcomes.
- Predictor agents equipped with tools (activation oracle, natural-language autoencoder, sparse autoencoder) performed no better than agents that only read the transcript.
- Claude Opus 4.8 was used as the predictor agent in the main experiments.
- Models trained on CHIVE data to predict how prompt edits change behavior showed substantial improvement in held-out settings.
- The negative result for interpretability tools held across variations, including two target models, three predictor model families, hyperparameter sweeps, and manual and automated attempts to elicit better tool use.
Context
According to Anthropic Alignment, the goal of evaluating explanations is to determine if they help predict what a model will do on related counterfactual inputs. For example, an explanation like "Gemma makes this coding error because it’s misled by the parameter names" should predict that renaming parameters would prevent the error. The CHIVE pipeline automatically generates the necessary datasets by pairing model behaviors with proposed explanations and informative counterfactuals.
Anthropic frequently uses interpretability tools, such as those studied in this work, in their system card evaluations. The authors note that their evaluation is a proxy, as ground truth is obtained by sampling the model, which differs from system card use cases where ground truth is often unavailable. Additionally, the behaviors discovered by CHIVE are generally simpler than the more sophisticated behaviors typically studied in system cards.
Why It Matters
This research provides a critical assessment of current interpretability tools, suggesting that their utility in predicting how prompt edits change LLM behavior may be limited. For builders and researchers, this implies that relying solely on these tools for understanding and predicting model responses in dynamic, real-world scenarios may not yield significant advantages over careful transcript analysis.
What To Do
- Note that current interpretability tools, including activation oracles and autoencoders, did not improve prediction of counterfactual outcomes in this evaluation.
- Consider that models trained on CHIVE data can generalize to predict behavioral changes from prompt edits.
- Review the methodology for evaluating explanations through counterfactual simulatability.
- Watch for further research on interpretability tools that explicitly address causal relationships rather than just correlational cues.