可参考博客

Pytorch可视化模型任意中间层的类激活热力图(Grad-CAM)_潜行隐耀的博客-CSDN博客_pytorch热力图

Pytorch输出网络中间层特征可视化_Joker-Tong的博客-CSDN博客_输出网络中间特征图

GitHub - utkuozbulak/pytorch-cnn-visualizations: Pytorch implementation of convolutional neural network visualization techniques

keras可视化中间层特征_joyce_peng的博客-CSDN博客_中间层特征可视化

图像处理特征可视化方法总结（特征图、卷积核、类可视化CAM）

神经网络之特征图可视化_AI bro的博客-CSDN博客_特征图的可视化

PyTorch模型训练特征图可视化（TensorboardX） - 知乎

图像特征可视化方法总结

（1）特征图可视化

特征图可视化有两类方法，一类是直接将某一层的feature map映射到0-255的范围，变成图像。另一类是使用一个预训练的反卷积网络（反卷积、反池化）将feature map变成图像，从而达到可视化feature map的目的。

（2）卷积核可视化

卷积的过程就是特征提取的过程，每一个卷积核代表着一种特征。如果图像中某块区域与某个卷积核的结果越大，那么该区域就越“像”该卷积核。基于以上的推论，如果我们找到一张图像，能够使得这张图像对某个卷积核的输出最大，那么我们就说找到了该卷积核最感兴趣的图像。

（3）类别激活可视化(Class Activation Mapping，CAM)

CAM（Class Activation Mapping，类别激活映射图），亦称为类别热力图或显著性图。它的大小与原图一致，像素值表示原始图片的对应区域对预测输出的影响程度，值越大贡献越大。目前常用的CAM系列包括：CAM、Grad-CAM、Grad-CAM++。

（4）注意力特征可视化

与CAM类似，只不过每个特征图所占权重来自于注意力，而不是最后层的全连接，基于注意力的特征可视化方法近年有比较多的研究。

（5）一些技术工具

tensorflow框架提供了模型和特征可视化的工具tensorboard，可使用pytorch框架引入。

tfrom torch.utils.tensorboard import SummaryWriter

更多使用细节参考PyTorch模型训练特征图可视化（TensorboardX） - 知乎

github代码

GitHub - utkuozbulak/pytorch-cnn-visualizations: Pytorch implementation of convolutional neural network visualization techniques

Gradient visualization with vanilla backpropagation
Gradient visualization with guided backpropagation [1]
Gradient visualization with saliency maps [4]
Gradient-weighted class activation mapping [3] (Generalization of [2])
Guided, gradient-weighted class activation mapping [3]
Score-weighted class activation mapping [15] (Gradient-free generalization of [2])
Element-wise gradient-weighted class activation mapping [16]
Smooth grad [8]
CNN filter visualization [9]
Inverted image representations [5]
Deep dream [10]
Class specific image generation [4] [14]
Grad times image [12]
Integrated gradients [13]
Layerwise relevance propagation [17]

GitHub - ZhugeKongan/TorchCAM: CAM', 'ScoreCAM', 'SSCAM', 'ISCAM' 'GradCAM', 'GradCAMpp', 'SmoothGradCAMpp', 'XGradCAM', 'LayerCAM' using by PyTorch.

下载包：

# python >= 3.6# Stable release
# You can install the last stable release of the package using pypi as follows:
pip install torchcam# or using conda:
conda install -c frgfm torchcam# Developer installation
# Alternatively, if you wish to use the latest features of the project that haven't made their way to a release yet, you can install the package from source:
git clone https://github.com/frgfm/torch-cam.git
pip install -e torch-cam/.

使用：

# CAM
# Learning Deep Features for Discriminative Localization: the original CAM paper
# https://arxiv.org/abs/1512.04150from torchvision.models import resnet18
from torchcam.cams import CAM
model = resnet18(pretrained=True).eval()
cam = CAM(model, 'layer4', 'fc')
with torch.no_grad(): out = model(input_tensor)
cam(class_idx=100)#Please note that by default, the layer at which the CAM is retrieved is set to the last non-reduced convolutional layer. If you wish to investigate a specific layer, use the target_layer argument in the constructor.# ScoreCAM
# paper:Score-CAM:Score-Weighted Visual Explanations for Convolutional Neural Networks
# https://arxiv.org/pdf/1910.01279.pdffrom torchvision.models import resnet18
from torchcam.cams import ScoreCAM
model = resnet18(pretrained=True).eval()
cam = ScoreCAM(model, 'layer4', 'fc')
with torch.no_grad(): out = model(input_tensor)
cam(class_idx=100)# SSCAM
# paper:SS-CAM: Smoothed Score-CAM for Sharper Visual Feature Localization
# https://arxiv.org/pdf/2006.14255.pdffrom torchvision.models import resnet18
from torchcam.cams import SSCAM
model = resnet18(pretrained=True).eval()
cam = SSCAM(model, 'layer4', 'fc')
with torch.no_grad(): out = model(input_tensor)
cam(class_idx=100)# ISCAM
# paper:IS-CAM: Integrated Score-CAM for axiomatic-based explanations
# https://arxiv.org/pdf/2010.03023.pdffrom torchvision.models import resnet18
from torchcam.cams import ISCAM
model = resnet18(pretrained=True).eval()
cam = ISCAM(model, 'layer4', 'fc')
with torch.no_grad(): out = model(input_tensor)
cam(class_idx=100)# GradCAM
# paper:Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
# https://arxiv.org/pdf/1610.02391.pdffrom torchvision.models import resnet18
from torchcam.cams import GradCAM
model = resnet18(pretrained=True).eval()
cam = GradCAM(model, 'layer4')
scores = model(input_tensor)
cam(class_idx=100, scores=scores)# Grad-CAM++
# paper:Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks
# https://arxiv.org/pdf/1710.11063.pdffrom torchvision.models import resnet18
from torchcam.cams import  GradCAMpp
model = resnet18(pretrained=True).eval()
cam =  GradCAMpp(model, 'layer4')
scores = model(input_tensor)
cam(class_idx=100, scores=scores)# Smooth Grad-CAM++
# paper:Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Models
# https://arxiv.org/pdf/1908.01224.pdffrom torchvision.models import resnet18
from torchcam.cams import SmoothGradCAMpp
model = resnet18(pretrained=True).eval()
cam = SmoothGradCAMpp(model, 'layer4')
scores = model(input_tensor)
cam(class_idx=100, scores=scores)# XGradCAM
# paper:Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs
# https://arxiv.org/pdf/2008.02312.pdffrom torchvision.models import resnet18
from torchcam.cams import XGradCAM
model = resnet18(pretrained=True).eval()
cam = XGradCAM(model, 'layer4')
scores = model(input_tensor)
cam(class_idx=100, scores=scores)# LayerCAM
# paper:LayerCAM: Exploring Hierarchical Class Activation Maps for Localization
# http://mmcheng.net/mftp/Papers/21TIP_LayerCAM.pdffrom torchvision.models import resnet18
from torchcam.cams import LayerCAM
model = resnet18(pretrained=True).eval()
cam = LayerCAM(model, 'layer4')
scores = model(input_tensor)
cam(class_idx=100, scores=scores)# Retrieving the class activation map
# Once your CAM extractor is set, you only need to use your model to infer on your data as usual. If any additional information is required, the extractor will get it for you automatically.from torchvision.io.image import read_image
from torchvision.transforms.functional import normalize, resize, to_pil_image
from torchvision.models import resnet18
from torchcam.cams import SmoothGradCAMppmodel = resnet18(pretrained=True).eval()
cam_extractor = SmoothGradCAMpp(model)
# Get your input
img = read_image("path/to/your/image.png")
# Preprocess it for your chosen model
input_tensor = normalize(resize(img, (224, 224)) / 255., [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])# Preprocess your data and feed it to the model
out = model(input_tensor.unsqueeze(0))
# Retrieve the CAM by passing the class index and the model output
activation_map = cam_extractor(out.squeeze(0).argmax().item(), out)# If you want to visualize your heatmap, you only need to cast the CAM to a numpy ndarray:import matplotlib.pyplot as plt
# Visualize the raw CAM
plt.imshow(activation_map.numpy()); plt.axis('off'); plt.tight_layout(); plt.show()# Or if you wish to overlay it on your input image:import matplotlib.pyplot as plt
from torchcam.utils import overlay_mask# Resize the CAM and overlay it
result = overlay_mask(to_pil_image(img), to_pil_image(activation_map, mode='F'), alpha=0.5)
# Display it
plt.imshow(result); plt.axis('off'); plt.tight_layout(); plt.show()

可视化heatmap或者叠加原图：

# Retrieving the class activation map
# Once your CAM extractor is set, you only need to use your model to infer on your data as usual. If any additional information is required, the extractor will get it for you automatically.from torchvision.io.image import read_image
from torchvision.transforms.functional import normalize, resize, to_pil_image
from torchvision.models import resnet18
from torchcam.cams import SmoothGradCAMppmodel = resnet18(pretrained=True).eval()
cam_extractor = SmoothGradCAMpp(model)
# Get your input
img = read_image("path/to/your/image.png")
# Preprocess it for your chosen model
input_tensor = normalize(resize(img, (224, 224)) / 255., [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])# Preprocess your data and feed it to the model
out = model(input_tensor.unsqueeze(0))
# Retrieve the CAM by passing the class index and the model output
activation_map = cam_extractor(out.squeeze(0).argmax().item(), out)# If you want to visualize your heatmap, you only need to cast the CAM to a numpy ndarray:import matplotlib.pyplot as plt
# Visualize the raw CAM
plt.imshow(activation_map.numpy()); plt.axis('off'); plt.tight_layout(); plt.show()# Or if you wish to overlay it on your input image:import matplotlib.pyplot as plt
from torchcam.utils import overlay_mask
# Resize the CAM and overlay it
result = overlay_mask(to_pil_image(img), to_pil_image(activation_map, mode='F'), alpha=0.5)
# Display it
plt.imshow(result); plt.axis('off'); plt.tight_layout(); plt.show()

CAM Zoo：

This project is developed and maintained by the repo owner, but the implementation was based on the following research papers:

Learning Deep Features for Discriminative Localization: the original CAM paper
Grad-CAM: GradCAM paper, generalizing CAM to models without global average pooling.
Grad-CAM++: improvement of GradCAM++ for more accurate pixel-level contribution to the activation.
Smooth Grad-CAM++: SmoothGrad mechanism coupled with GradCAM.
Score-CAM: score-weighting of class activation for better interpretability.
SS-CAM: SmoothGrad mechanism coupled with Score-CAM.
IS-CAM: integration-based variant of Score-CAM.
XGrad-CAM: improved version of Grad-CAM in terms of sensitivity and conservation.
Layer-CAM: Grad-CAM alternative leveraging pixel-wise contribution of the gradient to the activation.