Anthropic Research has published a study on the scaling laws and interpretability of learning from repeated data in large language models. The research investigates the effects of data repetition, which can occur either intentionally to upweight high-quality data or unintentionally due to imperfect deduplication processes.
The study involved training a family of models where most data was unique, but a small fraction was repeated multiple times. Researchers observed a double descent phenomenon, where test loss increased during training due to repeated data. This phenomenon suggests a predictable range of repetition frequency can lead to severe performance degradation.
Key Points
- Repeating just 0.1% of training data 100 times made an 800M-parameter model perform as poorly as a 400M-parameter model.
- The study found a strong double descent phenomenon, where repeated data can cause test loss to increase midway through training.
- A specific range of repetition frequency can lead to severe performance degradation.
- Data repetition disproportionately damages copying and internal structures associated with generalization, such as induction heads.
- This damage provides a possible mechanism for a shift from generalization to memorization.
Context
According to Anthropic Research, large language models are often trained on vast datasets that may contain repeated data. This repetition can be intentional, aimed at emphasizing higher-quality data, or unintentional due to limitations in data deduplication at various granularities (sentence, paragraph, or document level). The paper notes that some prior work has reported negative performance effects from such repeated data.
To systematically study these effects and understand their mechanisms, the researchers trained models with a small, highly repeated data fraction alongside mostly unique data. They hypothesize that a middle range of repetition allows data to be memorized, consuming a large fraction of the model's capacity and leading to peak degradation. The study connects these observations to mechanistic interpretability work by showing that data repetition specifically harms generalization-related structures like induction heads.
Why It Matters
This research offers builders and researchers a deeper understanding of how data repetition impacts model performance and generalization. It highlights a critical trade-off between data utilization strategies and model capacity, suggesting that even small amounts of repeated data can have disproportionately large negative consequences for model efficacy.
What To Do
- Note the potential for performance degradation when training data contains repeated elements.
- Review data deduplication strategies to minimize unintentional data repetition.
- Consider the implications of intentional data repetition for upweighting, particularly in light of the observed double descent phenomenon.
- Watch for further research on mechanistic interpretability related to generalization and memorization in models.
Keep Exploring
/atlas/claude-family