Volumetric TSDF Fusion of RGB-D Images in Python

This is a lightweight python script that fuses multiple registered color and depth images into a projective truncated signed distance function (TSDF) volume, which can then be used to create high quality 3D surface meshes and point clouds. Tested on Ubuntu 16.04.

An older CUDA/C++ version can be found here.

Requirements

• Python 2.7+ with NumPy, PyCUDA, OpenCV, Scikit-image and Numba. These can be quickly installed/updated by running the following:

  pip install --user numpy opencv-python scikit-image numba
  

• [Optional] GPU acceleration requires an NVIDA GPU with CUDA and PyCUDA:

  pip install --user pycuda
  

Demo

This demo fuses 1000 RGB-D images from the 7-scenes dataset into a 405 x 264 x 289 projective TSDF voxel volume with 2cm resolution at about 30 FPS in GPU mode (0.4 FPS in CPU mode), and outputs a 3D mesh mesh.ply which can be visualized with a 3D viewer like Meshlab.

Note: color images are saved as 24-bit PNG RGB, depth images are saved as 16-bit PNG in millimeters.

python demo.py

Seen In

• 3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions (CVPR 2017)
• Semantic Scene Completion from a Single Depth Image (CVPR 2017)
• Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images (CVPR 2016)

References

• A Volumetric Method for Building Complex Models from Range Images (SIGGRAPH 1996)
• KinectFusion: Real-Time Dense Surface Mapping and Tracking (ISMAR 2011)
• Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images (CVPR 2013)

Citing

This repository is a part of 3DMatch Toolbox. If you find this code useful in your work, please consider citing:

@inproceedings{zeng20163dmatch,
    title={3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions},
    author={Zeng, Andy and Song, Shuran and Nie{\ss}ner, Matthias and Fisher, Matthew and Xiao, Jianxiong and Funkhouser, Thomas},
    booktitle={CVPR},
    year={2017}
}