Anthropic Research has initiated a pilot program to provide external researchers with access to aggregate, real-world usage data from Claude. This initiative, detailed in a recent post, involved three research groups designing independent studies using Anthropic Insights, a privacy-preserving analysis tool. The data collection was performed by Anthropic on behalf of the researchers, who then conducted their own analyses. The pilot ran earlier this year, with the goal of understanding the impact of AI on people and society by making more data available to researchers, policymakers, and the public.
Key Points
- Anthropic ran a pilot program earlier this year, providing external researchers with access to aggregate, real-world Claude usage data.
- Three research groups designed studies for Anthropic Insights, a privacy-preserving analysis tool, with data collection run by Anthropic.
- The pilot involved roughly 250,000 Claude.ai or Claude Code conversations from April-May 2026.
- Participating groups included the Social and Language Technologies (SALT) Lab at Stanford University, the Human Information Processing Lab at the University of Oxford, and METR.
- Anthropic's contractual review rights were limited to user privacy, usage policy violations, confidential information, and research accuracy, ensuring researcher independence.
- The company is publicly releasing the aggregate data from each project.
- The pilot was resource-intensive and slower than typical internal AI lab research due to privacy and independence considerations.
Context
According to Anthropic Research, the pilot aimed to address the current concentration of real-world AI interaction data within a few labs. Researchers outside these labs typically rely on published analyses, which may not align with their specific questions, or public datasets that often reflect casual use rather than how most people use AI. The Anthropic Insights tool, formerly named 'Clio', is used internally by Anthropic teams to analyze usage patterns across millions of Claude conversations. For the pilot, an additional privacy audit was conducted on all data shared with third-party researchers to verify privacy protections, as detailed in an Appendix.
Why It Matters
This pilot represents an effort to enable independent research on AI usage by providing external access to real-world data, which can inform a broader understanding of AI's societal impact and foster more diverse research questions beyond those typically pursued by AI labs. Builders and researchers can observe a model for how AI companies might share data while maintaining privacy and researcher independence.
What To Do
- Review the high-level results shared from the three research groups to understand initial findings on human-AI collaboration, user sentiment, and productivity gains.
- Note the challenges Anthropic encountered in scaling the program, particularly regarding resource intensity and adapting internal research methods for external partners.
- Consider the implications of using Claude's judgments within Anthropic Insights for categorizing conversations and the potential for misrepresentation if questions are poorly phrased.
- Watch for the full write-up from the Human Information Processing Lab, which will be linked when public.