lyuwenyu/RT-DETR

English | 简体中文

RT-DETR: DETRs Beat YOLOs on Real-time Object Detection

---

This is the official implementation of papers

• DETRs Beat YOLOs on Real-time Object Detection
• RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer

Fig

🚀 Updates

• [2025.11.18] Release the newest member of the RT-DETR family: RT-DETRv4:Painlessly Furthering Real-Time Object Detection with Vision Foundation Models. By harnessing the rapidly evolving capabilities of Vision Foundation Models (VFMs), we boost lightweight detectors and, without incurring any extra inference latency, significantly improve the performance of the full-size model.
• [2024.11.28] Add torch tool for parameters and flops statistics. see run_profile.py
• [2024.10.10] Add sliced inference support for small object detecion. #468
• [2024.09.23] Add ✅Regnet and DLA34 for RTDETR.
• [2024.08.27] Add hubconf.py file to support torch hub.
• [2024.08.22] Improve the performance of ✅ RT-DETRv2-S to 48.1 mAP (+1.6 compared to RT-DETR-R18).
• [2024.07.24] Release ✅ RT-DETRv2!
• [2024.02.27] Our work has been accepted to CVPR 2024!
• [2024.01.23] Fix difference on data augmentation with paper in rtdetr_pytorch #84.
• [2023.11.07] Add pytorch ✅ rtdetr_r34vd for requests #107, #114.
• [2023.11.05] Upgrade the logic of remap_mscoco_category to facilitate training of custom datasets, see detils in Train custom data part. #81.
• [2023.10.23] Add discussion for deployments, supported onnxruntime, TensorRT, openVINO.
• [2023.10.12] Add tuning code for pytorch version, now you can tuning rtdetr based on pretrained weights.
• [2023.09.19] Upload ✅ pytorch weights convert from paddle version.
• [2023.08.24] Release RT-DETR-R18 pretrained models on objects365. 49.2 mAP and 217 FPS.
• [2023.08.22] Upload ✅ rtdetr_pytorch source code. Please enjoy it!
• [2023.08.15] Release RT-DETR-R101 pretrained models on objects365. 56.2 mAP and 74 FPS.
• [2023.07.30] Release RT-DETR-R50 pretrained models on objects365. 55.3 mAP and 108 FPS.
• [2023.07.28] Fix some bugs, and add some comments. 1, 2.
• [2023.07.13] Upload ✅ training logs on coco.
• [2023.05.17] Release RT-DETR-R18, RT-DETR-R34, RT-DETR-R50-m（example for scaled).
• [2023.04.17] Release RT-DETR-R50, RT-DETR-R101, RT-DETR-L, RT-DETR-X.

📣 News

• RTDETR and RTDETRv2 are now available in Hugging Face Transformers. #413, #549
• RTDETR is now available in ultralytics/ultralytics.

📍 Implementations

• 🔥 RT-DETRv2
  • paddle: code&weight
  • pytorch: code&weight
• 🔥 RT-DETR
  • paddle: code&weight
  • pytorch: code&weight
• Community ports
  • Android (LiteRT GPU): code&weight — RT-DETRv2-S on the LiteRT CompiledModel GPU with a Kotlin sample app, ~615 ms/frame on a Pixel 8a (still-image)

Model	Input shape	Dataset	$AP^{val}$	$AP^{val}_{50}$	Params(M)	FLOPs(G)	T4 TensorRT FP16(FPS)
RT-DETR-R18	640	COCO	46.5	63.8	20	60	217
RT-DETR-R34	640	COCO	48.9	66.8	31	92	161
RT-DETR-R50-m	640	COCO	51.3	69.6	36	100	145
RT-DETR-R50	640	COCO	53.1	71.3	42	136	108
RT-DETR-R101	640	COCO	54.3	72.7	76	259	74
RT-DETR-HGNetv2-L	640	COCO	53.0	71.6	32	110	114
RT-DETR-HGNetv2-X	640	COCO	54.8	73.1	67	234	74
RT-DETR-R18	640	COCO + Objects365	49.2	66.6	20	60	217
RT-DETR-R50	640	COCO + Objects365	55.3	73.4	42	136	108
RT-DETR-R101	640	COCO + Objects365	56.2	74.6	76	259	74
RT-DETRv2-S	640	COCO	48.1 (+1.6)	65.1	20	60	217
RT-DETRv2-M*	640	COCO	49.9 (+1.0)	67.5	31	92	161
RT-DETRv2-M	640	COCO	51.9 (+0.6)	69.9	36	100	145
RT-DETRv2-L	640	COCO	53.4 (+0.3)	71.6	42	136	108
RT-DETRv2-X	640	COCO	54.3	72.8 (+0.1)	76	259	74

Notes:

• COCO + Objects365 in the table means finetuned model on COCO using pretrained weights trained on Objects365.

🦄 Performance

🏕️ Complex Scenarios

🌋 Difficult Conditions

Citation

If you use RT-DETR or RTDETRv2 in your work, please use the following BibTeX entries:

@misc{lv2023detrs,
      title={DETRs Beat YOLOs on Real-time Object Detection},
      author={Yian Zhao and Wenyu Lv and Shangliang Xu and Jinman Wei and Guanzhong Wang and Qingqing Dang and Yi Liu and Jie Chen},
      year={2023},
      eprint={2304.08069},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

@misc{lv2024rtdetrv2improvedbaselinebagoffreebies,
      title={RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer}, 
      author={Wenyu Lv and Yian Zhao and Qinyao Chang and Kui Huang and Guanzhong Wang and Yi Liu},
      year={2024},
      eprint={2407.17140},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2407.17140}, 
}