VCG-team/DiffSegmenter
VCG-team/DiffSegmenter has 59 GitHub stars and 2 forks. GitHot measured 0 stars in 24 hours and 0 in 7 days. Last checked 2026-11-17 15:58 UTC. Last push 2026-07-05.
Measured star growth
- in 24h
- 0
- in 7 days
- 0
- in 15 days
- Update delayed
- in 1 month
- Update delayed
Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.
About
(TIP 2025, CCF-A) Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmenter
AIREADME.md
OriginalDiffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmenter
By Jinglong Wang, Xiawei Li, Jing zhang, Qiangyuan Xu, Qin Zhou, Qian Yu, Sheng Lu, Dong Xu.
This repository is an official implementation of the paper Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmenter. And you're welcome to our project page.
Follow-up Work Accepted by TIP 🎉
We are thrilled to share that our follow-up work, "ELBO-T2IAlign: A Generic ELBO-Based Method for Calibrating Pixel-level Text-Image Alignment in Diffusion Models", also has been accepted by IEEE Transactions on Image Processing (TIP), a CCF-A journal.
Building upon this repository, ELBO-T2IAlign investigates the impact of the original diffusion loss function and provides new insights into pixel-level text-image alignment and downstream tasks in diffusion models. Check it out here.
Citing DiffSegmenter
If you find DiffSegmenter useful in your research, please consider citing:
@misc{wang2023diffusion,
title={Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmenter},
author={Jinglong Wang and Xiawei Li and Jing Zhang and Qingyuan Xu and Qin Zhou and Qian Yu and Lu Sheng and Dong Xu},
year={2023},
eprint={2309.02773},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Installation
Requirements
Linux, CUDA>=11.7, GCC>=9.4
Python>=3.8
We recommend you to use Anaconda to create a conda environment:
conda create -n ldm python=3.8Then, activate the environment:
conda activate ldmOther requirements
pip install -r requirements.txt
Usage
Dataset preparation
Please download datasets and organize them as following:
├── COCO2014
│ ├── annotations
│ ├── coco_seg_anno
│ ├── images
│ │ ├── test2014
│ │ ├── train2014
│ │ └── val2014
│ └── mask
│ ├── train2014
│ └── val2014
└── VOCdevkit
├── VOC2010
│ ├── Annotations
│ ├── ImageSets
│ │ ├── Action
│ │ ├── Layout
│ │ ├── Main
│ │ ├── Segmentation
│ │ └── SegmentationContext
│ ├── JPEGImages
│ ├── SegmentationClass
│ ├── SegmentationClassContext
│ └── SegmentationObject
└── VOC2012
├── Annotations
├── ImageSets
│ ├── Action
│ ├── Layout
│ ├── Main
│ └── Segmentation
├── JPEGImages
├── SegmentationClass
├── SegmentationClassAug
└── SegmentationObject
Open Vocabulary Semantic Segmentation
Evaluation
For the setting of Open Vocabulary Semantic Segmentation, our model does not require training; it directly produces segmentation results.
The ‘open_vocabulary’ folder contains code for open vocabulary semantic segmentation. It includes scripts for the voc, coco, and Pascal context datasets.
Taking the voc10 dataset as an example:
Step 1: Modify your dataset path in the Python file.
Step 2: Run ptp_stable_voc10.py to generate segmentation results.
python ptp_stable_voc10.py
Step 3: Run the evaluation script, remember to update the file path. MIoU will be recorded in eval.txt
python evaluation_voc10.py