DensePoseFnL

Making DensePose fast and light

Ruslan Rakhimov*, Emil Bogomolov*, Alexandr Notchenko, Fung Mao, Alexey Artemov, Denis Zorin, Evgeny Burnaev

_{* equal contribution · Skolkovo Institute of Science and Technology · Huawei Moscow Research Center · New York University}

IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021

[ 📄 Paper | 🤗 Checkpoints ]

This repository contains the official implementation of the paper.

📖 Abstract

DensePose estimation task is a significant step forward for enhancing user experience computer vision applications ranging from augmented reality to cloth fitting. Existing neural network models capable of solving this task are heavily parameterized and a long way from being transferred to an embedded or mobile device. To enable Dense Pose inference on the end device with current models, one needs to support an expensive server-side infrastructure and have a stable internet connection. To make things worse, mobile and embedded devices do not always have a powerful GPU inside. In this work, we target the problem of redesigning the DensePose R-CNN model's architecture so that the final network retains most of its accuracy but becomes more light-weight and fast. To achieve that, we tested and incorporated many deep learning innovations from recent years, specifically performing an ablation study on 23 efficient backbone architectures, multiple two-stage detection pipeline modifications, and custom model quantization methods. As a result, we achieved 17× model size reduction and 2× latency improvement compared to the baseline model.

📝 Citation

@inproceedings{rakhimov2021making,
  title={Making DensePose fast and light},
  author={Rakhimov, Ruslan and Bogomolov, Emil and Notchenko, Alexandr and Mao, Fung and Artemov, Alexey and Zorin, Denis and Burnaev, Evgeny},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
  pages={1869--1877},
  year={2021}
}

📰 Changelog

• [October 2026] Checkpoints moved to Hugging Face.
• [October 2026] Added LiteRT export of CSE models that runs in realtime on Android CPU+GPU, along with the densepose-android app.
• [January 2022] Updated to latest Detectron2 and released the weights. Breaking: network quantization lost in this version, see issue.
• [June 2020] Initial code release

⚙️ Installation

# Install Detectron2 and DensePose
git clone https://github.com/facebookresearch/detectron2.git && cd detectron2
git checkout bb96d0b01d0605761ca182d0e3fac6ead8d8df6e
pip install -e .
cd projects/DensePose
pip install -e .

• timm==0.4.12
• torch==1.10.1

The torch==1.10.1 (CUDA 11.3) build has kernels for GPUs up to Ampere (sm_86).

🚀 Training and Evaluation

# Train
python train_net.py --config-file configs/mobile_parsing_rcnn_b_s3x.yaml --num-gpus 8

# Test
python train_net.py --config-file configs/mobile_parsing_rcnn_b_s3x.yaml --eval-only MODEL.WEIGHTS models/mobile_parsing_rcnn_b_s3x.pth

📦 Model Zoo

Name	lr sched	box AP	segm AP	dp. AP GPS	dp. AP GPSm	download
Mobile-Parsing-RCNN-B	s3x	57.1	59.0	50.4	54.4	model
Mobile-Parsing-RCNN-B-WC2M	s3x	59.4	63.7	57.3	60.3	model
Mobile-Parsing-RCNN-B-CSE	s3x	60.2	64.3	59.0	61.2	model

Download one or all of them with the Hugging Face CLI:

hf download rusrakhimov/DensePoseFnL config.yaml mobile_parsing_rcnn_b_cse_s3x.pth --local-dir models
hf download rusrakhimov/DensePoseFnL --local-dir models   # everything

WC2M corresponds to new training procedure and the model that performs estimation of confidence in regressed UV coordinates as well as confidences associated with coarse and fine segmentation; see Sanakoyeu et al., 2020 for details.

CSE corresponds to a continuous surface embeddings model for humans; see Neverova et al., 2020 for details.

Note: weights for Mobile-Parsing-RCNN-B (s3x) are not the same as presented in the paper but with a similar performance.

📚 More instructions

See DensePose (Getting Started)

⚖️ License

The code is released under the MIT license, see LICENSE. The checkpoints are released under Apache-2.0; the model card lists the terms of the data they were trained on.