jacobgil/vit-explain
Vit Explain
Vit-explain 是一个诊断框架，旨在解释视觉 Transformer（Vision Transformer）模型的决策过程。它作为一个用于检查内部模型状态的工具包，允许用户映射视觉注意力并分析特定图像特征如何影响分类结果。
该项目通过提供事后（post-hoc）模型解释脱颖而出，无需架构修改或重新训练即可分析训练好的神经网络。它采用诸如基于 Hook 的特征提取等技术在正向传递期间拦截内部激活，并结合 Transformer 注意力展开（attention rollout）和层级相关性传播（layer-wise relevance propagation）等方法，追踪信息从输入到输出的流动。
通过生成追踪注意力流和权重分布的热力图，该框架可视化了哪些特定像素和模式驱动了模型的预测。它通过使用目标类梯度过滤注意力图来支持类特定的解释，为调试深度学习模型和识别导致分类结果的视觉元素提供了诊断实用程序。
Features
- Deep Learning Interpretability - Analyzes neural network decision-making processes by mapping visual attention weights and gradients to input image regions.
- Transformer Explainability - Analyzes how vision transformer models process visual data by generating heatmaps that track attention flow across image regions.
- Attention Rollout Techniques - Aggregates attention maps across multiple layers by recursively multiplying weight matrices to trace the flow of information.
- Attention Visualizations - Generates heatmaps that track the flow of attention across image regions to reveal model processing priorities.
- Computer Vision Model Debugging - Visualizes internal decision-making processes by highlighting specific pixels and patterns that drive classification results.
- Internal Activation Hooks - Intercepts internal tensor activations during the forward pass to capture intermediate attention maps without modifying model architecture.
- Attribution Gradients - Calculates feature importance by backpropagating gradients from target classification outputs to input image patches.
- Relevance Propagation Methods - Distributes classification scores backward through network layers to identify which specific input pixels contributed most to the final prediction.
- Class-Specific Explainers - Filters attention maps with target class gradients to isolate visual elements contributing to specific classification outcomes.
- Transformer Explainability Tools - Provides diagnostic utilities for visualizing attention maps and gradients to interpret vision transformer classification features.
- Architecture Interpretation - Analyzes trained model weights and gradients to provide visual explanations without requiring retraining or architectural changes.
