worldbench/awesome-embodied-data-pyramid
worldbench/awesome-embodied-data-pyramid has 210 GitHub stars and 8 forks. Star changes are not measured yet. Last checked 2026-11-14 06:59 UTC. Last push 2026-11-11.
Measured star growth
- in 24h
- Collecting history
- in 7 days
- Collecting history
- in 15 days
- Collecting history
- in 1 month
- Collecting history
Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.
About
🔥 Data Pyramid for Embodied Manipulation: A Survey
Creative & MediaREADME.md
Original
Awesome Embodied Data Pyramid
This repository accompanies a survey on the data pyramid for robotics and embodied AI. It curates the datasets, data-collection paradigms, simulators, and data-hungry model families reviewed in the survey — spanning real-robot data, UMI (in-the-wild) data, egocentric / ego-exo human data, simulation data, and broad general (web-scale) data — together with the VLA / world-action / embodied-VLM models that consume them. Each entry links to its paper, project page, and code where available.
News 🔥🔥🔥
This project is actively maintained and continuously updated to make embodied data resources easier for the open-source community to discover, access, and download.
- 2026-10-30: Added FineART to real-robot datasets and updated the existing Daimon-Infinity / TacClaw-Infinity UMI entry with its paper and Hugging Face download link; distinguished the paper's 10,000+ hours from the dataset card's first 1,000-hour release.
- 2026-10-21: Added 11 paper/resource records on gaze-aligned teleoperation, human vision-motion-touch capture, human-to-robot data conversion, and simulation data generation; retained explicit release, access, and asset-license limits.
- 2026-10-11: Added 12 verified literature/resource entries across real-robot data, UMI engineering, egocentric supervision, simulation generation, and data curation; separated collected scale from released scale and consolidated World-Env / RehearseVLA.
- 2026-08-22: Added 20 resources from a July–August 2026 arXiv sweep, spanning contact-rich and industrial real-robot datasets (PRISM, SoftVTBench, RoboInter-Data), EgoData Engineering (7 new human-to-robot conversion works), and visuo-tactile world models used as simulators (ViTacWorld, N0-TWAM, OVTF).
- 2026-08-10: Added N0-Foundation and its OpenNeoData release to the real-robot data collection.
- 2026-08-07: Updated the data accumulation figure; introduced the EgoData Engineering collection with Ego2Robot and EgoEngine; and added the CS2-10k and RekaDaily-10k datasets from Reka AI.
- 2026-07-27: Open-sourced the project website and this GitHub project. You are welcome to use the curated links throughout this repository to access and download the collected resources. Issues and pull requests are encouraged to help keep these resources accessible and useful.
We sincerely thank all open-source dataset creators and contributors for their valuable work and continued support of the community. You can also use task-based search on the project website to find and download datasets.
If you have suggestions for new resources, improvements to methodologies, or corrections for broken links, please don't hesitate to open an issue or submit a pull request. Contributions of all kinds are welcome and greatly appreciated.
Table of Contents
Data Sources
Real-Robot Data
Egocentric & Ego-Exo Data
Simulation Data
Embodied Foundation Models
Related Surveys
Repository
The Data Pyramid: Data Sources
Real-Robot Data
Table: Recent real-robot datasets
Table: Real-robot datasets (2024 and earlier)
Data curation. FAKTUAL: Diversity You Can Actually Measure uses trajectory-kernel entropy to select diverse demonstration subsets without policy training or online rollouts. Diversity selection does not guarantee demonstration quality.
UMI Data
Table: UMI-style robot-free and cross-embodiment datasets
Collection and post-training extensions.
- PrimeBot Household also collects approximately 1,000 hours of bimanual UMI data. The official release provides staged UMI subsets; the collected total should not be reported as the downloadable total or as additional real-robot hours.
- HIL-UMI uses policy-guided handheld collection and advantage-conditioned post-training on one robot across four tasks. Its reported 5.63x collection throughput on the table-cleanup task is relative to real-robot HG-DAgger, not conventional UMI collection. Project; public code and a standalone dataset were not verified.
- UNI extends robot-free capture to wheeled navigation with a rollator and LiDAR phone (37.2 km / 10.8 hours). This is a navigation extension rather than a manipulation dataset. Project; code, weights, and the privacy-processed dataset remain pending release.
Egocentric & Ego-Exo Data
Table: EgoData Engineering
Release and supervision notes (checked 2026-10-11). HuRo's conversion code is available, but its full derived corpus is pending and third-party components restrict commercial use of the complete pipeline. EgoWild2Dex's code and data are announced rather than verified downloads. Ego-Exo4D-HM releases reconstruction-derived mesh annotations under CC BY 4.0; access to the upstream raw videos remains subject to Ego-Exo4D terms. Its 104.6 hours describe distinct reconstructed motion, whereas about 523 video hours count five camera streams. None of these annotation or retargeting pipelines automatically supplies physically executable, contact-consistent robot demonstrations.
Table: Recent egocentric and ego-exo datasets
Table: Egocentric and ego-exo datasets (2024 and earlier)
October 2026 availability notes (checked 2026-10-21). EgoExplorer reports 100 collected hours, while its official data card currently describes a one-hour mini release requiring manual access approval under CC BY-NC 4.0; the capture/reconstruction toolkit is still scheduled for release. HumanVerse reports 500 collected hours, but the code repository excludes the full mid-training datasets and complete deployment workflow; its linked sample endpoint was inaccessible in this check. EgoXperience labels its code, EgoX-Data-6M, benchmark, and weights as coming soon. React/ACTS announces code and data upon acceptance. These statuses do not establish that the full collected corpora can be downloaded.
Simulation Data
Curation note. For peer-reviewed work,
YearandPublication / Typeuse the formal publication year and venue. For arXiv-only work,Yearis the first public preprint year. Software without a canonical paper is labeledSoftware. The sections below distinguish reusable 3D/scene assets, simulation backends, evaluation benchmarks, and both downloadable synthetic datasets and systems that generate such data.
3D Assets & Scene Datasets
Table: 3D assets and scene datasets
Simulation & Rendering Backends
Table: Simulation and rendering backends
Benchmarks
Table: Simulation benchmarks
Benchmark access. Evo-DexBench separates its Apache 2.0 code/documentation from third-party asset terms; some PartNet assets require upstream access approval. A code license does not grant rights to every bundled or downloaded asset.
Simulation/Synthetic Datasets & Data-Generation Systems
Table: Simulation/synthetic datasets and data-generation systems
World Models as Simulators
Table: World models as simulators
General Data
Visual Data
Table: Visual data
Segmentation & Localization
Table: Segmentation and localization data
Video & Temporal Data
Table: Video and temporal data
Planning / Task Decomposition
Table: Planning and task decomposition data
| Year | Acronym | Paper | Venue | Project | Repo@GitHub |
|---|---|---|---|---|---|
| 2026 | SCOPE (planner-feedback data generation) | SCOPE: Scene-graph Construction Optimized by Plan Error-correction for Robot Long-term Planning Datasets | arXiv / planning-data generation | Project | Not verified released |
| 2023 | EgoPlan-IT | EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning | IJCV |  |
 |
| 2023 | EmbodiedGPT | EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought | NeurIPS | |
|
| 2023 | LLaRP | Large Language Models as Generalizable Policies for Embodied Tasks | ICLR | |
|
| 2022 | EgoTaskQA | EgoTaskQA: Understanding Human Tasks in Egocentric Videos | NeurIPS | |
|
| 2025 | WAP | World-aware Planning Narratives Enhance Large Vision-Language Model Planner | NeurIPS | |
|
Grasp Data
Table: Grasp data
| Year | Acronym | Paper | Venue | Project | Repo@GitHub |
|---|---|---|---|---|---|
| 2026 | MapleGrasp | MapleGrasp: Mask-guided Feature Pooling for Language-driven Efficient Robotic Grasping | WACV | |
|
| 2024 | Language-Driven 6-DoF Grasp Detection Using Negative Prompt Guidance | ECCV | |
|
|
| 2024 | DexGraspNet 2.0 | DexGraspNet 2.0: Learning Generative Dexterous Grasping in Large-scale Synthetic Cluttered Scenes | CoRL | |
|
| 2024 | Grasp-Anything | Grasp-Anything: Large-scale Grasp Dataset from Foundation Models | ICRA | |
|
| 2021 | SuctionNet-1Billion | SuctionNet-1Billion: A Large-Scale Benchmark for Suction Grasping | RAL | |
|
| 2018 | Jacquard | Jacquard: A Large Scale Dataset for Robotic Grasp Detection | IROS | |
|
| 2020 | GraspNet-1Billion | GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping | CVPR | |
|
Embodied Foundation Models
Representative VLA and WAM Methods
Representative VLA and WAM methods by release time, model type, institution, and data source.
Data sources: 
Table: Representative VLA and WAM methods (74 entries)
| Time | Method | Institution | Project | Model | Data |
|---|---|---|---|---|---|
| 2023.3 | PaLM-E | |
VLA | |
|
| 2023.7 | RT-2 | DeepMind | |
VLA | |
| 2024.5 | Octo | UC Berkeley | |
VLA | |
| 2024.6 | OpenVLA | Stanford University | |
VLA | |
| 2024.10 | GR-2 | ByteDance | |
VLA | |
| 2024.10 | π0 | Physical Intelligence | |
VLA | |
| 2024.10 | RDT-1B | THU | |
VLA | |
| 2024.11 | CogACT | THU | |
VLA | |
| 2025.1 | SpatialVLA | Shanghai AI Lab | |
VLA | |
| 2025.1 | UP-VLA | THU | |
VLA | |
| 2025.3 | HybridVLA | PKU | |
VLA | |
| 2025.3 | GR00T N1 | NVIDIA | |
VLA | |
| 2025.3 | GO-1 | Shanghai AI Lab & AgiBot | |
VLA | |
| 2025.3 | CoT-VLA | NVIDIA | |
VLA | |
| 2025.4 | UWM | University of Washington | |
WAM | |
| 2025.4 | π0.5 | Physical Intelligence | |
VLA | |
| 2025.5 | UniVLA | HKU | |
VLA | |
| 2025.6 | SmolVLA | Hugging Face | |
VLA | |
| 2025.6 | GR00T N1.5 | NVIDIA | |
VLA | |
| 2025.7 | DreamVLA | SJTU | |
VLA | |
| 2025.7 | EgoVLA | UC San Diego | |
VLA | |
| 2025.7 | Being-H0 | PKU | |
VLA | |
| 2025.7 | GR-3 | ByteDance Seed | |
VLA | |
| 2025.7 | H-RDT | THU | |
VLA | |
| 2025.8 | GalaxeaVLA(G0) | Galaxea | |
VLA | |
| 2025.8 | ReconVLA | HKUST | |
VLA | |
| 2025.8 | MolmoAct | AI2 & UW | |
VLA | |
| 2025.8 | EO-1 | Shanghai AI Lab | |
VLA | |
| 2025.9 | RynnVLA-001 | DAMO | |
VLA | |
| 2025.10 | X-VLA | THU | |
VLA | |
| 2025.10 | InternVLA-M1 | Shanghai AI Lab | |
VLA | |
| 2025.10 | VITRA | THU | |
VLA | |
| 2025.11 | METIS | PKU | |
VLA | |
| 2025.11 | iFlyBot-VLA | iFLYTEK & LindenBot | |
VLA | |
| 2025.12 | VideoVLA | XJTU | |
VLA | |
| 2025.12 | Motus | THU | |
WAM | |
| 2025.12 | GR00T N1.6 | NVIDIA | |
VLA | |
| 2026.1 | InternVLA-A1 | Shanghai AI Lab | |
VLA | |
| 2026.1 | LaST_0 | PKU | |
VLA | |
| 2026.1 | Being-H0.5 | BeingBeyond | |
VLA | |
| 2026.1 | LingbotVLA | Ant Group | |
VLA | |
| 2026.1 | LingbotVA | Ant Group | |
WAM | |
| 2026.2 | RDT2 | THU | |
VLA | |
| 2026.2 | Xiaomi-Robotics-0 | Xiaomi | |
VLA | |
| 2026.2 | ABot-M0 | AMAP CV Lab | |
VLA | |
| 2026.2 | DM0 | Dexmal | |
VLA | |
| 2026.2 | LDA-1B | PKU | |
WAM | |
| 2026.2 | DreamZero | NVIDIA | |
WAM | |
| 2026.2 | EgoScale | NVIDIA | |
VLA | |
| 2026.3 | GigaWorld-Policy | GigaAI | |
WAM | |
| 2026.3 | MolmoB0T | AI2 | |
VLA | |
| 2026.3 | UniDex | THU | |
VLA | |
| 2026.4 | JoyAI-RA 0.1 | JD | |
VLA | |
| 2026.4 | π0.7 | Physical Intelligence | |
VLA | |
| 2026.4 | GR00T N1.7 | NVIDIA | |
VLA | |
| 2026.4 | Being-H0.7 | BeingBeyond | |
WAM | |
| 2026.4 | MotuBrain | Shengshu | |
WAM | |
| 2026.5 | Qwen-VLA | Qwen | |
VLA | |
| 2026.5 | Wall-OSS-0.5 | X Square Robot | |
VLA | |
| 2026.5 | MolmoAct2 | UW & AI2 | |
VLA | |
| 2026.6 | Wall-WM | X Square Robot | |
WAM | |
| 2026.6 | LaST-HD | PKU | |
VLA | |
| 2026.6 | Hy-Embodied-0.5-VLA | Tencent Robotics X | |
VLA | |
| 2026.6 | Kairos | ACE Robotics | |
WAM | |
| 2026.6 | Qwen-RobotManip | Qwen | |
VLA | |
| 2026.7 | ABot-M0.5 | AMAP CV Lab | |
WAM | |
| 2026.7 | ACE-Brain-0.5 | ACE-Robotics | |
VLA | |
| 2026.7 | InternVLA-A1.5 | Shanghai AI Lab | |
VLA | |
| 2026.7 | LingbotVLA 2.0 | Ant Group | |
VLA | |
| 2026.7 | LingbotVA 2.0 | Ant Group | |
WAM | |
| 2026.7 | HumanScale | PKU | |
WAM | |
| 2026.7 | GigaWorld-Policy-0.5 | GigaAI | |
WAM | |
| 2026.7 | Xiaomi-Robotics-1 | Xiaomi | |
VLA | |
| 2026.7 | Xiaomi-Robotics-U0 | Xiaomi Robotics | |
WAM | |
Related Surveys
These surveys provide background on robot learning and simulation-to-real transfer.
Table: Related surveys
| Year | Scope | Paper | Venue | Project | Repo@GitHub |
|---|---|---|---|---|---|
| 2026 | World models for robot learning | World Model for Robot Learning: A Comprehensive Survey | arXiv | |
|
| 2025 | Sim-to-real reinforcement learning | A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models | arXiv | |
Contributing
Contributions are welcome! Please open an issue or pull request to add resources, fix links, update metadata, or suggest improvements. For new resources, please include the paper title, project page, code link if available, release date, and most relevant category. For datasets, distinguish collected, processed, and publicly released scale. Include the checked paper version/date and any known access or license restrictions, and mark unavailable or unverified resources explicitly.
Acknowledgement
If you find this repository useful, please consider citing our paper:
@misc{ye2026datapyramid,
title={{Data Pyramid for Embodied Manipulation: A Survey}},
author={Yifan Ye and Yankai Fu and Yaoxu Lv and Bohan Hou and Jun Cen and Lingdong Kong and Duo Zheng and Tianxing Chen and Jiaming Liu and Ziang Cao and Yunfan Lou and Wei Chow and Xian Sun and Yingshuo Wang and Kuangzhi Ge and Xiaowei Chi and Xidong Zhang and Zhibo Pang and Yiwu Zhong and Sirui Han and Zhihe Lu and Weihao Yuan and Qifeng Chen and Michael Yu Wang and Yao Mu and Ziwei Liu and Jianfei Yang and Ping Luo and Shanghang Zhang},
year={2026},
eprint={2607.24744},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2607.24744}
}
Explore related projects
ytdl-org/youtube-dl
Command-line program to download videos from YouTube.com and other video sites
141,560 starsexcalidraw/excalidraw
Virtual whiteboard for sketching hand-drawn like diagrams
135,658 starsgodotengine/godot
Godot Engine – Multi-platform 2D and 3D game engine
120,113 starsmrdoob/three.js
JavaScript 3D Library.
118,345 stars