Anthropic Research has published details on Project Pilot, a research initiative exploring how AI models can interact with the physical world by autonomously flying drones. This project, conducted with Andon Labs, introduces Drone-Bench, a new benchmark designed to test an AI model's capability to locate and follow a person using a drone.
Key Points
- Project Pilot investigates AI models' ability to operate hardware, specifically drones, for tasks like aerial surveillance.
- Drone-Bench is a new benchmark created by Andon Labs to evaluate AI agents' control of drones for surveillance tasks.
- The core task involves an AI model controlling a quad-rotor drone in an indoor office to locate and follow a person.
- This task requires complex sub-tasks, including developing control schema, mapping space, finding a target from a reference photo, and following the target.
- Andon Labs decomposed the main goal into five necessary sub-tasks for evaluation.
- The evaluations were reproduced in software to allow for multiple, faster runs compared to physical experiments.
- A human-AI team at Andon Labs established a baseline for performance, demonstrating end-to-end success for the sub-tasks.
Context
According to Anthropic Research, Project Pilot builds on previous work like Project Vend, where AI models ran a small shop, and Project Fetch, which examined robots as intermediaries between digital models and physical objects. The research aims to understand how frontier models interact with the physical world, noting that operating hardware is a capability expected to become broadly accessible to AI models. This work is part of Anthropic's broader effort, including its Frontier Red Team, to measure AI capabilities and understand the benefits and risks of autonomous AI operation of robots.
Why It Matters
This research highlights the dual-use nature of AI models and drone technology, presenting both opportunities for applications like search and rescue or disaster response, and risks of potential misuse. The development of benchmarks like Drone-Bench provides a structured way to assess and track the progress of AI in controlling physical systems, informing discussions among technology developers, civil society, and governments regarding effective norms and governance frameworks.
What To Do
- Note the distinction between the Drone-Bench baseline and unassisted human capability or the ceiling of human-AI collaboration.
- Watch for further publications from Anthropic Research or Andon Labs detailing model performance against the Drone-Bench baseline.
- Consider the implications of AI models approaching or exceeding this baseline for human oversight in drone operations.
- Observe how the identified sub-tasks contribute to the overall capability of autonomous drone control.