Anthropic has launched a $5 million grant program aimed at fostering independent research into how AI systems influence user wellbeing. The program will offer direct funding, access to Anthropic's models, and technical support to grantees. The goal is to facilitate the creation of open-source evaluations that can help the AI industry measure the impact of models on their users. Grantees will operate independently and publish their work as open-source projects for broader developer use.
Key Points
- Anthropic is launching a $5 million grant program.
- The program funds independent research into AI's impact on user wellbeing.
- Grantees will receive direct funding, access to Anthropic's models, and technical support.
- The research will focus on building open-source evaluations.
- Applications are due by September 21.
- Selected applicants for full proposals will be notified by October 5.
Context
According to Anthropic, AI systems have become integral to how many people work, learn, and solve problems, also serving as conversational partners and sources of emotional support. However, the industry is still developing clear standards for model behavior in sensitive conversations, such as when users seek companionship or navigate a mental health crisis. Evaluating wellbeing is complex, requiring more context than assessing a single answer's accuracy, as user distress or specific histories (e.g., disordered eating) can make a seemingly reasonable response inappropriate or harmful.
Why It Matters
This initiative aims to address the nuanced and significant stakes involved in developing AI safeguards that evolve with models and their uses. By funding independent evaluations and benchmarks, Anthropic seeks to engage experts like clinicians, psychologists, and methodologists in this emerging field, contributing to more rigorous and context-aware AI development practices.
What To Do
- Review Anthropic's guidance on rigorous wellbeing evaluations.
- Consider the specific criteria for evaluations, such as clearly stating measurements and involving clinical experts.
- Note the emphasis on testing both precautions and harms, and reflecting multi-turn conversations.
- Submit applications by September 21 if interested in the grant program.