NVIDIA's Developer Blog details how World Action Models (WAMs), which are built on video world models, offer stronger physical generalization for robot policies compared to traditional Vision-Language-Action (VLA) models. The open NVIDIA Cosmos 3 model, based on a Mixture-of-Transformers architecture, serves as a foundation for post-training WAM-based robot policies.
WAMs enable zero-shot transfer to new tasks, robots, and environments by leveraging learned dynamics rather than relying solely on semantic mappings. This approach addresses a central challenge in robotics: building policies that generalize beyond their training demonstrations.
Key Points
- WAMs are built on video world models, while VLAs typically use vision-language models (VLMs).
- The NVIDIA Cosmos 3 model is an omni-model world foundation built on a Mixture-of-Transformers (MoT) architecture.
- Cosmos 3 was trained on a multimodal dataset including roughly 767 million images, 348 million videos, and 8 million action samples.
- Cosmos 3 comes in three sizes: 4B NVIDIA Cosmos Edge, 16B NVIDIA Cosmos Nano, and 64B NVIDIA Cosmos 3 Super.
- WAMs can provide practical benefits such as requiring less task-specific data for adaptation and offering better behavior outside the training distribution.
- Deployment tiers for Cosmos 3 range from high-throughput workstation serving to real-time on-device inference with NVIDIA Jetson hardware.
- Cosmos3-Nano-Policy-DROID is a 16B-parameter policy post-trained from Cosmos 3 Nano for the DROID platform.
Context
According to the NVIDIA Developer Blog, a key limitation of VLA models is that their VLM backbones are optimized to describe the world, not to predict how it evolves. This means VLAs generalize well semantically but are less effective at physical generalization to unseen behaviors and environments. WAMs overcome this by building policies on a video world model, which inherently models how the world evolves, providing a physics prior that reduces the need to teach dynamics from scratch during post-training.
Why It Matters
This shift from VLAs to WAMs, particularly with models like NVIDIA Cosmos 3, offers builders a path to developing more robust and adaptable robot policies. By focusing on world dynamics rather than just semantic understanding, WAMs can lead to policies that require less data for training, perform better in novel situations, and are easier to adapt to different robot embodiments, potentially reducing development costs and accelerating deployment.
What To Do
- Review the NVIDIA Cosmos 3 model's architecture and training data to understand its foundation for WAMs.
- Consider post-training a WAM from Cosmos 3 with your own data to compare its performance against existing VLA-based policies.
- Explore the different deployment tiers offered by Cosmos 3 (Edge, Nano, Super) to match your specific hardware and performance requirements.
- Investigate the Cosmos3-Nano-Policy-DROID and Cosmos3-Edge-Policy-DROID examples for practical application insights.
Keep Exploring
/atlas/image-models