vipshop/cache-dit

**🤗Why Cache-DiT❓❓**Cache-DiT is built on top of the 🤗Diffusers library and now supports nearly ALL DiTs from Diffusers. It provides hybrid cache acceleration (DBCache, TaylorSeer, SCM, etc.) and comprehensive parallelism optimizations, including Context Parallelism, Tensor Parallelism, hybrid 2D or 3D parallelism, and dedicated extra parallelism support for Text Encoder, VAE, and ControlNet.

Cache-DiT is compatible with compilation, offload, and quantization, fully integrates with SGLang Diffusion, vLLM-Omni, TensorRT-LLM, xDiT, and runs natively on NVIDIA GPUs, Huawei Ascend NPUs and AMD GPUs. Cache-DiT is fast, easy to use, and flexible for various DiTs (online docs at 📘cache-dit.io).

pip install -U cache-dit # PyPI, stable release.
pip install git+https://github.com/vipshop/cache-dit.git # latest.

>>> import cache_dit
>>> from diffusers import DiffusionPipeline
>>> pipe = DiffusionPipeline.from_pretrained(...).to("cuda")
>>> cache_dit.enable_cache(pipe) # Cache Acceleration with One-line code.
>>> from cache_dit import DBCacheConfig, ParallelismConfig
>>> cache_dit.enable_cache( # Or, Hybrid Cache Acceleration + Parallelism.
...   pipe, cache_config=DBCacheConfig(), # w/ default
...   parallelism_config=ParallelismConfig(ulysses_size=2))
>>> from cache_dit import DBCacheConfig, ParallelismConfig, QuantizeConfig
>>> cache_dit.enable_cache( # Or, Hybrid Cache + Parallelism + Quantization.
...   pipe, cache_config=DBCacheConfig(), # w/ default
...   parallelism_config=ParallelismConfig(ulysses_size=2),
...   quantize_config=QuantizeConfig(quant_type=...))
>>> output = pipe(...) # Then, just call the pipe as normal.

# Required: CUDA >=13.0, PyTorch >=2.11 (a CUDA 13 build), GLIBC 2.38.
pip install -U cache-dit --extra-index-url https://vipshop.github.io/cache-dit/whl/cu13/ 
# Latest, from source: one flag builds every CUDA extension (SVDQuant, comm, attn).
git submodule update --init --recursive --force # the SVDQuant sources need spdlog
CACHE_DIT_BUILD_CUDA=1 pip install -e ".[quantization]" # default, all archs.

>>> from cache_dit import QuantizeConfig
>>> pipe = DiffusionPipeline.from_pretrained(...).to("cuda")
>>> # ✅ PTQ: "...{dtype}_r{rank}", required `calibrate_fn`.
>>> def calibrate_fn(**_: object) -> None:
...  for prompt in calibration_prompts: _ = pipe(...)
>>> pipe.transformer = cache_dit.quantize(
...  pipe.transformer, quant_config=QuantizeConfig(
...  quant_type="svdq_nvfp4_r32", calibrate_fn=calibrate_fn,
...  serialize_to="path"))
>>> pipe.transformer = cache_dit.load(pipe.transformer, "path")

>>> # ✅ DQ: "...{dtype}_r{rank}_dq", NO calibration need.
>>> pipe.transformer = cache_dit.quantize(
...   pipe.transformer, quant_config=QuantizeConfig(
...   quant_type="svdq_{int4|nvfp4}_r{32|64|128|256|...}_dq",
...   svdq_kwargs={"smooth_strategy": "few_shot"})) 
>>> output = pipe(...) # Then, just call the pipe as normal.

>>> import cache_dit
>>> cache_dit.layerwise_offload(
...   pipe, # nn.Module: pipe, transformer, text_encoder, etc.
...   onload_device="cuda",
...   offload_device="cpu",
...   async_transfer=True,
...   transfer_buckets=4,
...   persistent_buckets=64,
...   persistent_bins=8,
...   prefetch_limit=True,
...   max_copy_streams=4,
...   max_inflight_prefetch_bytes="8gib")
>>> output = pipe(...) # Then, just call the pipe as normal.

🚪Supported DiT Models

Cache-DiT supports 60+ DiT pipeline families (150+ Variants) from 🤗Diffusers, covering the vast majority of DiT-based pipelines. For full support matrix and detailed usage, please refer to our documentation at 📘cache-dit.io.

🤖Agentic Workflows

Cache-DiT provides a model-integration SKILL to help users integrate new DiT pipelines into Cache-DiT, including Cache, CP, TP, TE-P, VAE-P and carefully designed test cases. Users can use it with Coding Agents, e.g, GitHub Copilot, Claude Code, Open Code.

gh skill install vipshop/cache-dit cache-dit-model-integration --allow-hidden-dirs

Model integration: The SKILL guides the Coding Agent through: search and download the model weights (e.g. from 🤗Hugging Face or ModelScope), match the model against the corresponding 🤗Diffusers pipeline and transformer code, and then implement support for it following the SKILL's instructions.

/cache-dit-model-integration "Please download and support MiniMax-H3 in Cache-DiT."

Performance: It can also run the performance matrix for models that Cache-DiT already supports. For examples:

/cache-dit-model-integration "Run the perf matrix for MiniMax-H3 with 1/2/4/8 GPUs."

🌐Community Integration

• 🎉MindIE-SD x Cache-DiT
• 🎉xDiT x Cache-DiT
• 🎉ComfyUI x Cache-DiT
• 🎉Diffusers x Cache-DiT
• 🎉TensorRT-LLM x Cache-DiT
• 🎉SGLang Omni x Cache-DiT
• 🎉SGLang Diffusion x Cache-DiT
• 🎉vLLM-Omni x Cache-DiT
• 🎉Nunchaku x Cache-DiT
• 🎉SD.Next x Cache-DiT
• 🎉stable-diffusion.cpp x Cache-DiT
• 🎉jetson-containers x Cache-DiT
• 🎉RH-MinMax-H3 x Cache-DiT

©️Acknowledgements

Special thanks to vipshop's Computer Vision AI Team for supporting testing and deployment of this project. We learned and reused codes from: Diffusers, SGLang, vLLM-Omni, Nunchaku, xDiT and TaylorSeer.

©️Citations

@misc{cache-dit@2025,
  title={Cache-DiT: A PyTorch-native Inference Engine with Cache, Parallelism, Quantization and CPU Offload for DiTs.},
  url={https://github.com/vipshop/cache-dit.git},
  note={Open-source software available at https://github.com/vipshop/cache-dit.git},
  author={DefTruth, vipshop.com, etc.},
  year={2025}
}