Physics-aware generation A Newton's cradle in motion: momentum transfer rendered with physical fidelity	Transfer World-scenario control layout → photoreal driving video	Action policy Policy run: "put the screwdriver and the glove in the purple container"
Simulation Generate synthetic data and create simulation for autonomous driving	Action-conditioned World Model Forward dynamics: egocentric rollout from input camera + hand pose	World reasoning Reason in complex real-world scenarios: a rolling ball means a child or pet may follow

---

What's new

• [Oct 2026]  Interactive World Simulation with Cosmos for Physical AI - example model checkpoints and training recipes: HF collection · Cookbook · Technical report · Website.
• [Jul 2026] Cosmos3-Edge released - the 4B tier for on-device, real-time deployment (Jetson AGX Orin / Thor / RTX Pro 6000).
• [May 2026] Cosmos 3 released: HF collection · Technical Report.

What is Cosmos?

NVIDIA Cosmos is an open platform for building physical AI applications, supporting industry verticals such as robotics, autonomous vehicles, smart infrastructure, manufacturing, healthcare, agriculture. The mission of Cosmos is to provide better data, better environment, better starting point, and better tooling for physical AI developers.

Cosmos 3, the current model family, is a suite of omnimodal world models built on a unified Mixture-of-Transformers architecture (technical report).

One model, two towers:

	Inputs	Outputs	Use it for
Reasoner	text, vision	text	world understanding, grounding, task planning, embodied reasoning
Generator	text, vision, audio, action	vision, audio, action	world simulation, future prediction, synthetic data, policy learning

This repository is the landing repo to start your physical AI journey with Cosmos. Here you can find detailed information on Cosmos models and explore Cosmos cookbook - a set of agent runnable recipes for various objectives and industries.

The recipes are supported by the Cosmos platform, including Cosmos Framework for training infra (SFT, LoRA, RL, distillation etc.), Cosmos Curator for data curation, and Cosmos Evaluator for model output evaluation.

Find your path

I want to…	Go to	Time
See it work - zero install	Video generation, visual reasoning	1 min
Generate my first video	Quickstart ↓	10 min
Run generation/reasoning inference	Inference recipes	10 min
Serve an OpenAI-compatible API	Serving guide: vLLM, vLLM-Omni, or NIM	30 min
Post-train on my own data: SFT, LoRA, distill	Post-train and distill recipes	hours
Explore agent-runnable notebooks	Cookbook	browse
Evaluate a model	Evaluation suites: PAIBench, Physics-IQ, VLMEvalKit	hours
Check latency & throughput	Performance benchmarks	browse

Generate your first video

Before running the code, request access to nvidia/Cosmos-1.0-Guardrail and accept its access conditions. Once access is granted, log in below with a Hugging Face read token from the same account. Logging in alone does not grant access.

uv venv --python 3.13 --seed --managed-python && source .venv/bin/activate
uv pip install --torch-backend=auto \
  "diffusers @ git+https://github.com/huggingface/diffusers.git" \
  accelerate av cosmos_guardrail huggingface_hub imageio imageio-ffmpeg \
  torch torchvision transformers
uvx hf@latest auth login   # Authenticate for the gated Guardrail repository

import torch
from diffusers import Cosmos3OmniPipeline
from diffusers.utils import export_to_video

pipe = Cosmos3OmniPipeline.from_pretrained(
    "nvidia/Cosmos3-Nano", torch_dtype=torch.bfloat16, device_map="cuda"
)
video = pipe(prompt="A mobile robot navigates a warehouse aisle and stops at a shelf.").video
export_to_video(video, "first_video.mp4", fps=24)

First run downloads the 16B checkpoint; diffusion steps are compute-heavy, so long step times are normal. Full options, image/sound modes, and every other backend: audiovisual cookbooks · setup issues: environment setup guide.

Models

Cosmos 3 ships as three base models. Every deployment tier, one omnimodal architecture:

Base model	Size	Runs on	Best for
Cosmos3-Super	64B	H200 / B200 / GB200	Highest quality; synthetic data generation; teacher for distillation
Cosmos3-Nano	16B	RTX Pro 6000 / H100 / B200	Balanced speed and quality; strong base model to post-train
Cosmos3-Edge	4B	Jetson AGX Orin / Thor / RTX Pro 6000	Edge deployment; real-time robot policy and visual reasoning

Example checkpoints in the cosmos3-examples collection are post-trained variants of the base models. They demonstrate what post-training with Cosmos Framework can specialize Cosmos for.

Example checkpoint	Base	Demonstrates
Cosmos3-Nano-Sim-Auto	Nano	Closed-loop driving simulation - multi-view + LiDAR
Cosmos3-Nano-Transfer-Auto	Nano	Multi-view driving SDG - up to 11 views + LiDAR
Cosmos3-Nano-Sim-Bimanual	Nano	Closed-loop interactive robot policy simulation
Cosmos3-Nano-Sim-Transfer	Nano	Auto-regressive transfer - depth, segmentation, edge, blur
Cosmos3-Super-Text2Image	Super	Elite quality text-to-image
Cosmos3-Super-Text2Image-4Step	Super	Elite quality text-to-image, 17-25x faster
Cosmos3-Super-Image2Video	Super	Elite quality image-to-video
Cosmos3-Super-Image2Video-4Step	Super	Elite quality image-to-video, 17-25x faster
Cosmos3-Nano-Policy-DROID	Nano	Open SOTA DROID robot policy, runs on RTX Pro 6000
Cosmos3-Edge-Policy-DROID	Edge	DROID robot policy at edge-deployable scale

Full I/O specs, generation settings, and supported action embodiments: model reference.

Platform

Project	Purpose
Cosmos	This repo
Cosmos Framework	Train, optimize, and deploy physical AI models, SFT · LoRA · distillation · RL post-training, for Cosmos and beyond
Cosmos Curator	Distributed data curation: processing, annotation, filtering, dedup
Cosmos Evaluator	Automated evaluation system for world generation & reasoning outputs

Cosmos 3 runs on Diffusers, Transformers, vLLM, vLLM-Omni, SGLang, TensorRT-LLM, and NIM. Pick a backend in the environment setup guide.

Community & contributing

Questions and ideas → Discussions. Bugs → Issues. Code and recipes → Cosmos Framework.

Limitations & safety

Cosmos 3 can produce artifacts in long, high-resolution, or physically complex outputs (temporal inconsistency, object morphing, implausible dynamics). Safety-critical applications need additional validation and system-level safety analysis. Generation ships with guardrails on by default.

Citation & license

@techreport{nvidia2026cosmos3,
  title  = {Cosmos 3: Omnimodal World Models for physical AI},
  author = {{NVIDIA Cosmos Team}},
  year   = {2026},
  url    = {https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf}
}

Source code and models are released under OpenMDW-1.1. Custom licensing: [email protected]. This project may download third-party open source software; review those licenses before use.