PaddlePaddle/InterpretDL
PaddlePaddle/InterpretDL has 262 GitHub stars and 41 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-18 14:49 UTC. Last push 2024-09-04.
实测星标增长
- 24小时
- 0
- 7天
- 正在收集历史
- 15天
- 正在收集历史
- 1个月
- 正在收集历史
各周期内的星标净变化,周期相互重叠,并非每日新增。缺失的测量保持空白。星标代表关注度,不代表软件质量。
关于
InterpretDL: Interpretation of Deep Learning Models,基于『飞桨』的模型可解释性算法库。
机器学习README_CN.md
原文中文 | ENGLISH

InterpretDL: 基于『飞桨』的模型可解释性算法库
InterpretDL(全称interpretations of deep learning models), 是基于飞桨 的模型可解释性算法库,其中集成了许多可解释性算法,包括LIME, Grad-CAM, Integrated Gradients等等,还添加和更新许多SOTA算法和最新的可解释性算法。
InterpretDL持续更新中,欢迎所有贡献。

为什么选择InterpretDL
随着深度学习模型的越发复杂,人们很难去理解其内部的工作原理。“黑盒”的可解释性正成为许多优秀研究者的焦点。InterpretDL不仅支持经典的可解释性算法,也对最新的可解释性算法持续更新。
通过这些非常有效的可解释性方法,人们可以更好知道模型为什么好,为什么不好,进而可以针对性提高模型性能。
对于正在开发新可解释性算法的研究者而言,利用InterpretDL和已有算法比较也非常方便。
:fire: :fire: :fire: News :fire: :fire: :fire:
(2022/04/27) 除了已提供的每个算法的具体使用案例,新增一个快速上手教程:可从 GitHub 或者 NBViewer 查看。我们也在准备更多的教程以方便InterpretDL的用户使用。教程和使用案例均可以在 tutorial 目录下查看。
(2022/01/06) 新增 Cross-Model Consensus Explanation 算法. 简单来说,这个可解释性算法将多个模型的解释结果进行平均,来定位数据中最具代表性的特征,并且定位的精度相比单个模型的结果更为准确。更多细节请查看文章。
Consensus: Xuhong Li, Haoyi Xiong, Siyu Huang, Shilei Ji, Dejing Dou. Cross-Model Consensus of Explanations and Beyond for Image Classification Models: An Empirical Study. arXiv:2109.00707.
我们准备了一个简单的demo,用了6个模型进行平均,原文中建议用15个以上的模型进行解释,可以得到一个更好的结果。实现代码可以参考使用案例。

Demo
可解释性算法对“黑盒”的预测做出了解释。
下表是使用一些可解释性算法对原始图片的可视化展示,告诉我们为什么模型做出了"bull_mastiff"的预测。
| Original Image | IntGrad (demo) | SG (demo) | LIME (demo) | Grad-CAM (demo) |
|---|---|---|---|---|
![]() |
![]() |
![]() |
![]() |
![]() |
文本情感分类任务中模型给出积极或消极预测的原因也可以被可视化。 这里是demo. 对中文的样本数据也同样适用。 这里是demo.

Contents
- InterpretDL: Interpretation of Deep Learning Models based on PaddlePaddle
- Why InterpretDL
- :fire: :fire: :fire: News :fire: :fire: :fire:
- Demo
- Contents
- Installation
- Documentation
- Usage Guideline
- Roadmap
- Copyright and License
- Recent News
Installation
安装百度「飞桨」深度学习框架 paddlepaddle, 建议选择CUDA支持版本。
Pip installation
pip install interpretdl
# or with tsinghua mirror
pip install interpretdl -i https://pypi.tuna.tsinghua.edu.cn/simple
Developer installation
git clone https://github.com/PaddlePaddle/InterpretDL.git
# ... fix bugs or add new features
cd InterpretDL && pip install -e .
# welcome to propose pull request and contribute
Unit Tests
# run gradcam unit tests
python -m unittest -v tests.interpreter.test_gradcam
# run all unit tests
python -m unittest -v
Documentation
在线链接: interpretdl.readthedocs.io.
或者下载到本地:
git clone https://github.com/PaddlePaddle/InterpretDL.git
cd docs
make html
open _build/html/index.html
Usage Guideline
所有解释器都继承类 Interpreter, 借助 interpret(**kwargs)
以调用。
# an example of SmoothGradient Interpreter.
import interpretdl as it
from paddle.vision.models import resnet50
paddle_model = resnet50(pretrained=True)
sg = it.SmoothGradInterpreter(paddle_model, use_cuda=True)
gradients = sg.interpret("test.jpg", visual=True, save_path=None)
使用细节在 tutorials 文件夹。
Roadmap
我门希望构建一个强大的模型解释工具库,并提供评估方式。 以下是我们已实现的算法,我们也将继续加入新的算法。 欢迎贡献算法,或者告诉我们想用的算法,我们将尽可能地实现。
已实现的算法
以输入特征为解释对象
- SmoothGrad
- IntegratedGradients
- Occlusion
- GradientSHAP
- LIME
- GLIME (LIMEPrior)
- NormLIME/FastNormLIME
- LRP
以中间过程特征为解释对象
- CAM
- GradCAM
- ScoreCAM
- Rollout
- TAM
数据层面的可解释性算法
- Forgetting Event
- SGDNoise
- TrainIng Data analYzer (TIDY)
跨模型的解释
- Consensus
计划中的算法
数据层面的可解释性算法
- Influence Function
评估方式
- Perturbation Tests
- Deletion & Insertion
- Localization Ablity
- Local Fidelity
- Sensitivity
Tutorials
我们计划为每种可解释性算法提供至少一个例子,涵盖CV和NLP的应用。
现有的教程在 tutorials 文件夹。
References of Algorithms
SGDNoise: On the Noisy Gradient Descent that Generalizes as SGD, Wu et al 2019IntegratedGraients: Axiomatic Attribution for Deep Networks, Mukund Sundararajan et al. 2017CAM,GradCAM: Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization, Ramprasaath R. Selvaraju et al. 2017SmoothGrad: SmoothGrad: removing noise by adding noise, Daniel Smilkov et al. 2017GradientShap: A Unified Approach to Interpreting Model Predictions, Scott M. Lundberg et al. 2017Occlusion: Visualizing and Understanding Convolutional Networks, Matthew D Zeiler and Rob Fergus 2013Lime: "Why Should I Trust You?": Explaining the Predictions of Any Classifier, Marco Tulio Ribeiro et al. 2016NormLime: NormLime: A New Feature Importance Metric for Explaining Deep Neural Networks, Isaac Ahern et al. 2019ScoreCAM: Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks, Haofan Wang et al. 2020ForgettingEvents: An Empirical Study of Example Forgetting during Deep Neural Network Learning, Mariya Toneva et al. 2019LRP: On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation, Bach et al. 2015Rollout: Quantifying Attention Flow in Transformers, Abnar et al. 2020TAM: Explaining Information Flow Inside Vision Transformers Using Markov Chain. Yuan et al. 2021Consensus: Cross-Model Consensus of Explanations and Beyond for Image Classification Models: An Empirical Study. Li et al 2021Perturbation: Evaluating the visualization of what a deep neural network has learned.Deletion&Insertion: RISE: Randomized Input Sampling for Explanation of Black-box Models.PointGame: Top-down Neural Attention by Excitation Backprop.
Copyright and License
InterpretDL 基于 Apache-2.0 license 提供。
Recent News
(2021/10/20) 新增 Transition Attention Maps (TAM) explanation method for PaddlePaddle Vision Transformers. 一如往常,几行代码即可调用! 详情查看 使用案例, 以及 paper:
TAM: Tingyi Yuan, Xuhong Li, Haoyi Xiong, Hui Cao, Dejing Dou. Explaining Information Flow Inside Vision Transformers Using Markov Chain. In Neurips 2021 XAI4Debugging Workshop.
import paddle
import interpretdl as it
# load vit model and weights
# !wget -c https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_base_patch16_224_pretrained.pdparams -P assets/
from assets.vision_transformer import ViT_base_patch16_224
paddle_model = ViT_base_patch16_224()
MODEL_PATH = 'assets/ViT_base_patch16_224_pretrained.pdparams'
paddle_model.set_dict(paddle.load(MODEL_PATH))
# Call the interpreter.
tam = it.TAMInterpreter(paddle_model, use_cuda=True)
img_path = 'samples/el1.png'
heatmap = tam.interpret(
img_path,
start_layer=4,
label=None, # elephant
visual=True,
save_path=None)
heatmap = tam.interpret(
img_path,
start_layer=4,
label=340, # zebra
visual=True,
save_path=None)
| image | elephant | zebra |
|---|---|---|
![]() |
![]() |
![]() |
- (2021/07/22) 新增 Rollout Explanations for PaddlePaddle Vision Transformers. 点击 notebook 浏览可视化结果。
import paddle
import interpretdl as it
# wget -c https://paddle-imagenet-models-name.bj.bcebos.com/dygraph/ViT_small_patch16_224_pretrained.pdparams -P assets/
from assets.vision_transformer import ViT_small_patch16_224
paddle_model = ViT_small_patch16_224()
MODEL_PATH = 'assets/ViT_small_patch16_224_pretrained.pdparams'
paddle_model.set_dict(paddle.load(MODEL_PATH))
img_path = 'assets/catdog.png'
rollout = it.RolloutInterpreter(paddle_model, use_cuda=True)
heatmap = rollout.interpret(img_path, start_layer=0, visual=True)







