Anthropic Research has published findings on scaling influence functions to large language models (LLMs). This work aims to provide greater visibility into model behavior by identifying which training examples contribute most to a given output. The research addresses the challenge of applying influence functions to LLMs, which traditionally struggle with the computational demands of inverse-Hessian-vector product (IHVP) calculations.
The team utilized the Eigenvalue-corrected Kronecker-Factored Approximate Curvature (EK-FAC) approximation to extend influence functions to LLMs with up to 52 billion parameters. This method achieved accuracy comparable to traditional influence function estimators while significantly reducing the computational cost of IHVP.
Key Points
- Influence functions aim to identify training examples that most contribute to a model's behavior.
- Scaling influence functions to large language models (LLMs) is challenging due to the computational cost of inverse-Hessian-vector product (IHVP).
- Anthropic used the Eigenvalue-corrected Kronecker-Factored Approximate Curvature (EK-FAC) approximation to scale influence functions to LLMs up to 52 billion parameters.
- EK-FAC achieved similar accuracy to traditional estimators while making IHVP computation orders of magnitude faster.
- Two algorithmic techniques, TF-IDF filtering and query batching, were used to reduce the cost of computing gradients for candidate training sequences.
- Influence functions were applied to investigate LLM generalization patterns, including sparsity, abstraction with scale, math and programming abilities, cross-lingual generalization, and role-playing.
- A limitation was identified where influences decay to near-zero when the order of key phrases is flipped.
Context
According to Anthropic Research, influence functions offer a counterfactual perspective: how model parameters and outputs would change if a specific sequence were added to the training set. While effective for smaller models, their application to LLMs has been limited by the intensive computation of the IHVP. The EK-FAC approximation addresses this by providing a more efficient method for this calculation, enabling the study of models with a significantly larger parameter count. The research also incorporated TF-IDF filtering and query batching to further optimize the gradient computation for training sequences.
Why It Matters
This research provides a new tool for understanding the generalization properties of large language models. For builders and researchers, the ability to trace model behavior back to specific training data can inform efforts in model interpretability, risk mitigation, and the development of more reliable AI systems. Identifying specific limitations, such as sensitivity to phrase order, can guide future model development and evaluation strategies.
What To Do
- Review the research paper to understand the technical details of the EK-FAC approximation.
- Note the identified limitation regarding phrase order sensitivity when evaluating LLM robustness.
- Consider how influence functions could be applied to analyze generalization patterns in other large models.
- Watch for further research on scaling influence functions to even larger or more complex model architectures.
Keep Exploring
/atlas/claude-family /techniques/system-user-separation /techniques/ptcf