News

• [2026.09] Released World2Motion for scene-aware 3D motion generation.
• [2026.09] Released Triangular Resampling for long-horizon streaming motion generation.
• [2026.09] Released FloodDiffusion 2 for path-controllable streaming motion generation.
• [2026.09] Released GestureFAR for streaming co-speech gesture generation.
• [2026.09] Intentional-Gesture accepted to NeurIPS 2026.
• [2026.05] Released PersonaGesture for single-reference speaker style personalization.
• [2026.02] FloodDiffusion accepted to CVPR 2026 (Highlight).
• [2025.07] GestureLSM accepted to ICCV 2025.

1. Demo

FD2 (2.3 ms Real-Time Stream)	Triangular Resampling (120s+ Long-Horizon Rollout)
World2Motion (GPT-6 + 3D World Grounding)	Co-Speech (Audio → Full-Body & 3D Avatar)

2. Models, Weights & Benchmarks

2.1 Generating Face and Body Animation from Speech (BEAT2)

All outputs are in SMPL-X (body + hands) and FLAME (face) parameters evaluated on BEAT2 (Speaker 2 Scott).

Models from character-policy Team

Models from Community*

*Agent-summarized models reported on BEAT2 (Speaker 2); may contain inaccuracies, issues or PRs for corrections and new entries are welcome.

2.2 Generating Body Animation from Text (HumanML3D)

All outputs are 3D body motion parameters evaluated on HumanML3D.

Models from character-policy Team

Models from Community*

*Agent-summarized models reported on HumanML3D; may contain inaccuracies, issues or PRs for corrections and new entries are welcome.

3. Data & Tools

Data & Tools from character-policy Team

4. For Agents Using This Repo

• Input: .wav mono speech audio (resampled automatically to 16,000 Hz).
• Output: .npz dictionary at 30 FPS (T = int(audio_seconds * 30)), saved via emage_utils.motion_io.beat_format_save:

git clone https://github.com/haiyangliu0/character-policy.git
cd character-policy

uv venv .venv --python 3.10
source .venv/bin/activate
uv pip install "setuptools<70" pip wheel "numpy<1.24"
uv pip install torch torchvision torchaudio librosa==0.9.2 soundfile scipy scikit-learn tqdm pyyaml wandb
uv pip install transformers==4.46.3 huggingface_hub accelerate einops timm
uv pip install smplx==0.1.28 trimesh pyrender pyopengl imageio moviepy==1.0.3 opencv-python wget
uv pip install --no-build-isolation chumpy

git clone https://huggingface.co/haiyangliu0/emage_evaltools

# Inference & rendering (.npz + .mp4 to ./examples/motion)
python test_emage_audio.py --audio_folder ./examples/audio --save_folder ./examples/motion --visualization
python test_camn_audio.py --audio_folder ./examples/audio --save_folder ./examples/motion --visualization --nopytorch3d
python test_disco_audio.py --audio_folder ./examples/audio --save_folder ./examples/motion --visualization --nopytorch3d

# Training on BEAT2
hf download haiyangliu0/BEAT2 --repo-type dataset --local-dir ./BEAT2
python ./datasets/foot_contact.py
torchrun --nproc_per_node=1 train_emage_audio.py --config ./configs/emage_audio.yaml --evaluation