Anthropic has released the second update from its Economic Index, an initiative designed to track AI's impact on labor markets and the economy. This report focuses on usage data from Claude.ai following the launch of Claude 3.7 Sonnet, which features strengths in agentic coding and an "extended thinking" mode.
The analysis covers 1 million anonymized conversations from Claude.ai Free and Pro users over 11 days after Claude 3.7 Sonnet's release. The model was the default on Claude.ai and its mobile app during this period.
Key Points
- The analysis covers 1 million anonymized conversations from Claude.ai Free and Pro users over 11 days following the launch of Claude 3.7 Sonnet.
- Claude 3.7 Sonnet's "extended thinking" mode is predominantly used in technical and creative problem-solving contexts.
- Tasks associated with computer and information research scientists show almost 10% usage of extended thinking, followed by software developers at around 8%.
- Digital creative roles, such as multimedia artists (~7%) and video game designers (~6%), also show substantial usage of extended thinking.
- Augmentation comprises 57% of usage, with learning interactions rising from ~23% to ~28%.
- Community and Social Service tasks, including education and guidance counseling, approach 75% augmentation.
- A new bottom-up taxonomy of 630 granular usage categories, ranging from "Help resolve household plumbing" to "Provide guidance on battery technologies," has been released.
Context
According to Anthropic, the analysis tool, Clio, maps each conversation to one of 17,000 tasks in the U.S. Department of Labor’s ONET database. This allows for the examination of overall patterns in occupations and high-level occupational categories associated with these tasks. The report notes modest increases in usage proportion for coding, education, and sciences, which could indicate AI diffusion, novel coding applications, or unexpected model capability improvements. The "extended thinking" mode allows the model to process complex questions for longer when activated by the user.
Why It Matters
This report offers insights into how users interact with advanced AI models like Claude 3.7 Sonnet, particularly its specialized features. Understanding the distribution of augmentation versus automation, and the specific contexts where new capabilities like "extended thinking" are applied, can inform developers and researchers about real-world model utility and areas for further development.
What To Do
- Review the full report for detailed findings on occupational usage patterns.
- Download the freely available datasets to explore the 630 granular usage categories.
- Examine the dataset mapping ONET tasks to their associated thinking mode fraction on Hugging Face.
- Compare the augmentation and automation data provided on Hugging Face at the task and occupation levels.
