# ai-dawang/plugnplay-modules

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/ai-dawang-plugnplay-modules).**

4,968 stars · 365 forks · Python

## Links

- GitHub: https://github.com/ai-dawang/PlugNPlay-Modules
- awesome-repositories: https://awesome-repositories.com/repository/ai-dawang-plugnplay-modules.md

## Description

PlugNPlay-Modules is a collection of reusable PyTorch computer vision modules and deep learning architectural components. It provides a library of standardized building blocks for constructing neural networks, focusing on attention mechanisms, signal processing layers, and feature fusion modules.

The project is distinguished by its extensive variety of attention primitives, covering spatial, channel, and temporal weighting, as well as specialized variants like deformable, frequency-enhanced, and linear-complexity attention. It also implements advanced signal processing tools within the neural network context, including discrete wavelet transforms, Laplacian pyramid decomposition, and frequency-domain filtering.

The codebase covers a broad surface of capabilities, including multi-scale feature extraction, hierarchical feature fusion, and various convolutional layer optimizations. It further includes utilities for tensor normalization, contrastive learning, and specialized loss functions for imbalanced datasets.

## Tags

### Artificial Intelligence & ML

- [Attention Mechanisms](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms.md) — Provides a comprehensive library of attention mechanism implementations, including spatial, channel, and deformable variants.
- [Hybrid Spatial-Channel Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/hybrid-spatial-channel-attention.md) — Integrates spatial and channel-wise weighting blocks to selectively emphasize important features.
- [Computer Vision](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/computer-vision.md) — Provides a comprehensive toolkit of reusable PyTorch libraries for deep learning image processing and computer vision tasks.
- [Channel Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/channel-attention.md) — Weights feature maps by learning importance across channels using global average pooling and linear transformations. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/DICAM.py))
- [Frequency-Domain Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/frequency-domain-attention.md) — Implements a channel attention mechanism that leverages Discrete Cosine Transforms to capture frequency-domain features. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/FECAttention.py))
- [Hybrid Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/hybrid-attention.md) — Ships a combination of channel-wise and spatial attention blocks to adaptively weight image information. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/HAAM.py))
- [Integrated Attention Blocks](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/integrated-attention-blocks.md) — Implements a single block combining spatial, channel, and gating attention to refine feature representations. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/EFF2d.py))
- [Linear-Complexity Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/linear-complexity-attention.md) — Implements linear-complexity attention by approximating the attention mechanism through weighted summations.
- [Standardized Convolutional Sequences](https://awesome-repositories.com/f/artificial-intelligence-ml/convolutional-neural-networks/one-dimensional-convolutions/standardized-convolutional-sequences.md) — Offers standardized convolutional sequences including batch normalization and activation to simplify model definition. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/FMS.py))
- [Depthwise Separable Convolutions](https://awesome-repositories.com/f/artificial-intelligence-ml/convolutional-neural-networks/pointwise-convolutions/depthwise-separable-convolutions.md) — Implements depthwise separable convolutions that split operations into depthwise and pointwise layers for efficiency. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/FCHiLo.py))
- [Deep Learning Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/deep-learning-architectures.md) — Offers standardized building blocks for constructing deep neural networks with multi-scale feature extraction and normalization.
- [Feature Fusion Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/feature-fusion-architectures.md) — Provides architectural blocks for aggregating multi-modal, multi-scale, and bitemporal data into unified representations. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/tree/main/%E7%89%B9%E5%BE%81%E8%9E%8D%E5%90%88))
- [Cross-Resolution Fusion](https://awesome-repositories.com/f/artificial-intelligence-ml/feature-fusion-architectures/cross-resolution-fusion.md) — Provides mechanisms to merge feature maps from different hierarchy levels using upsampling and attention-based bagging.
- [Laplacian Pyramid Decompositions](https://awesome-repositories.com/f/artificial-intelligence-ml/inference-scaling/resolution-scaling/hierarchical-feature-pyramids/multi-scale-feature-pyramids/laplacian-pyramid-decompositions.md) — Provides a process that generates a Laplacian pyramid by iteratively downsampling and subtracting tensors. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/FADConv.py))
- [Multi-Scale Feature Aggregation](https://awesome-repositories.com/f/artificial-intelligence-ml/inference-scaling/resolution-scaling/hierarchical-feature-pyramids/multi-scale-feature-pyramids/multi-scale-feature-aggregation.md) — Combines representations from different resolutions using parallel convolutional branches and adaptive weighting.
- [Computer Vision Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/computer-vision/computer-vision-architectures.md) — Provides structural designs and neural network layouts specifically engineered for processing and interpreting visual data.
- [1D Convolutional Layers](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/convolution-layers/1d-convolutional-layers.md) — Provides a set of reusable 1D convolutional architectural components for processing sequential signals. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/tree/main/1D%E6%A8%A1%E5%9D%97))
- [Configurable Convolutional Blocks](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/convolution-layers/convolutional-block-composers/configurable-convolutional-blocks.md) — Implements modular convolutional blocks with configurable normalization and activation functions. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/MLA.py))
- [Deformable Convolutions](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/convolution-layers/deformable-convolutions.md) — Implements convolutional layers with learnable offsets to adapt receptive fields to input geometry.
- [Partial Convolutions](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/convolution-layers/partial-convolutions.md) — Implements partial convolutions that apply filters to a subset of channels to reduce computational overhead. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/FMB.py))
- [Fast Fourier Convolutions](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/machine-learning-concepts/network-architectures-and-layers/image-convolutions/fast-fourier-convolutions.md) — Processes image features in the frequency domain using Fast Fourier Transforms to capture global dependencies.
- [Multi-Head Attention Mechanisms](https://awesome-repositories.com/f/artificial-intelligence-ml/multi-head-attention-mechanisms.md) — Implements the standard multi-head scaled dot-product attention mechanism using queries, keys, and values. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(arXiv%202023)%20ScaledDotProductAttention.py))
- [Activation-Free Blocks](https://awesome-repositories.com/f/artificial-intelligence-ml/activation-functions/gated-linear-units/activation-functions/activation-free-blocks.md) — Implements blocks that replace traditional nonlinear activation functions with depthwise convolutions and channel attention. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/NAF.py))
- [Attention Masking](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-masking.md) — Implements a gated mask to control information flow and refine feature representations within attention layers. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/LGAG.py))
- [Attention Gating](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/attention-gating.md) — Refines feature maps by combining gating signals with input tensors through grouped convolutions. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/EFF2d.py))
- [Bottleneck Channel Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/bottleneck-channel-attention.md) — Calculates channel-wise importance using global average pooling and a bottleneck structure. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/EFF2d.py))
- [Convolutional Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/convolutional-attention.md) — Uses average and max pooling to enhance important information specifically for image segmentation tasks. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/CPCA2d.py))
- [Deformable Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/deformable-attention.md) — Provides an attention mechanism with linear complexity and deformable offsets to focus on relevant spatial locations. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(CVPR%202022)%20DAT.py))
- [Double Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/double-attention.md) — Captures long-range spatial dependencies by gathering and redistributing global descriptors across the feature map. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/DA.py))
- [Dual Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/dual-attention.md) — Integrates spatial and channel attention mechanisms to capture global dependencies and channel-wise relationships. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/DA_Block.py))
- [Dynamic Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/dynamic-attention.md) — Computes channel, filter, spatial, and kernel attention weights to adaptively refine feature representations. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/FADConv.py))
- [Fine-Grained Channel Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/fine-grained-channel-attention.md) — Computes adaptive weights for feature channels using global average pooling and 1D convolutions. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/FCA.py))
- [Global Context Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/global-context-attention.md) — Computes attention across global features using orthogonal projection matrices and generalized kernels. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/ENLTB.py))
- [Hybrid Global-Local Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/hybrid-global-local-attention.md) — Combines local convolutional features with global transformer representations for efficient image patch processing. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(ICLR%202022)%20MobileViTAttention.py))
- [Parallel Attention Structures](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/parallel-attention-structures.md) — Implements a parallel combination of pixel-level, channel-level, and adaptive weight attention mechanisms. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/MixStructure.py))
- [Parameter-Free Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/parameter-free-attention.md) — Calculates attention weights based on feature map variance to emphasize regions without learnable parameters. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/BFAM.py))
- [Pixel-Level Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/pixel-level-attention.md) — Refines feature maps by weighting channels and spatial pixels to emphasize important regions. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/FFA.py))
- [Position-Aware Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/position-aware-attention.md) — Combines global channel-wise weighting and local spatial attention to emphasize important descriptors. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/CPAM.py))
- [Spatial Attention Weighting](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/spatial-attention-weighting.md) — Provides a spatial attention weighting module that uses average and max pooling to highlight critical image regions. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/EFF2d.py))
- [Temporal Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/spatio-temporal-attention/temporal-attention.md) — Provides temporal attention modules that use 1D convolutions and global average pooling to weight time-based features. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/DPTAM.py))
- [Temporal Attention Sequences](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/spatio-temporal-attention/temporal-attention-sequences.md) — Implements temporal attention sequences using shared multi-layer perceptrons and adaptive pooling. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/GAU.py))
- [Squeeze-and-Excitation Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/squeeze-and-excitation-attention.md) — Combines multi-scale convolutions and squeeze-and-excitation blocks to emphasize spatial and channel information. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(arXiv%202021)%20PSA.py))
- [Strip Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/strip-attention.md) — Processes spatial features through horizontal and vertical attention to capture long-range dependencies for image restoration. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/DSAM.py))
- [Wavelet-Domain Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms/wavelet-domain-attention.md) — Implements wavelet-domain attention that combines dual spatial-channel attention with wavelet transforms. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/HWAB.py))
- [Spatial-Channel Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/channel-processing/spatial-channel-attention.md) — Implements spatial-channel attention layers that refine multi-dimensional data using convolutional filters. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/GAU.py))
- [Poly-Kernel Inception Modules](https://awesome-repositories.com/f/artificial-intelligence-ml/convolutional-kernel-optimizations/poly-kernel-inception-modules.md) — Provides Poly Kernel Inception Blocks that process features through parallel depthwise convolutions with varying dilations. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(CVPR%202024)PKIBlock.py))
- [Inverted Residual Blocks](https://awesome-repositories.com/f/artificial-intelligence-ml/convolutional-neural-networks/pointwise-convolutions/depthwise-separable-convolutions/inverted-residual-blocks.md) — Provides inverted residual MLP structures with expansion phases using depthwise-separable convolutions. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/HFF.py))
- [Feature Recalibration Modules](https://awesome-repositories.com/f/artificial-intelligence-ml/feature-extraction/convolutional-feature-extractors/feature-map-aggregators/feature-recalibration-modules.md) — Implements combined channel and spatial attention mechanisms to recalibrate multiscale feature representations. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/LPA.py))
- [Dynamic Feature Fusion](https://awesome-repositories.com/f/artificial-intelligence-ml/feature-fusion-architectures/dynamic-feature-fusion.md) — Combines two input feature maps using global average pooling and learned attention weights. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/DFF2d.py))
- [Hierarchical Feature Fusion](https://awesome-repositories.com/f/artificial-intelligence-ml/feature-fusion-architectures/hierarchical-feature-fusion.md) — Combines pooling, attention mechanisms, and spatial convolutions to refine multi-scale representations for classification. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/HFF.py))
- [Global Filter Networks](https://awesome-repositories.com/f/artificial-intelligence-ml/global-filter-networks.md) — Implements a global filter network combining patch embedding and global filtering blocks for image classification. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(NeurIPS%202021)%20GFNet.py))
- [Patch Embedding Modules](https://awesome-repositories.com/f/artificial-intelligence-ml/image-convolution-operations/image-patch-embedders/patch-embedding-modules.md) — Provides patch embedding modules that convert 2D image data into flattened sequences for transformer architectures. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(NeurIPS%202021)%20GFNet.py))
- [Multi-Scale Feature Pyramids](https://awesome-repositories.com/f/artificial-intelligence-ml/inference-scaling/resolution-scaling/hierarchical-feature-pyramids/multi-scale-feature-pyramids.md) — Implements a feature pyramid network that aggregates hierarchical image representations to improve object detection accuracy. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/AFPN.py))
- [Adaptive Multi-Scale Fusion](https://awesome-repositories.com/f/artificial-intelligence-ml/inference-scaling/resolution-scaling/hierarchical-feature-pyramids/multi-scale-feature-pyramids/adaptive-multi-scale-fusion.md) — Combines multiple intermediate representation levels using adaptive weighting to integrate multi-scale information. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/FFA.py))
- [Self-Modulated Aggregators](https://awesome-repositories.com/f/artificial-intelligence-ml/inference-scaling/resolution-scaling/hierarchical-feature-pyramids/multi-scale-feature-pyramids/multi-scale-feature-aggregation/self-modulated-aggregators.md) — Provides a system for combining multi-scale feature information using adaptive pooling and variance-based modulation. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/FMB.py))
- [Spatially-Varying Convolutions](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/convolution-layers/deformable-convolutions/spatially-varying-convolutions.md) — Provides spatially-varying convolutions that modulate feature maps using coordinate-based gating and bilinear coefficient maps. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(BMVC%202023)CoordGate.py))
- [Layer Parameter Optimization](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/infrastructure/model-optimization-and-inference/training-algorithms/deep-learning-optimization/layer-parameter-optimization.md) — Implements computational efficiency improvements through separable and partial convolutions and stochastic depth.
- [Modular Layer Compositions](https://awesome-repositories.com/f/artificial-intelligence-ml/model-composition-architectures/hybrid-layer-compositions/modular-layer-compositions.md) — Provides a system for constructing neural architectures by stacking standardized modular blocks of convolutions and activations.
- [Spatially-Adaptive Modulators](https://awesome-repositories.com/f/artificial-intelligence-ml/model-training/spatial-control-networks/spatially-adaptive-normalization/spatially-adaptive-modulators.md) — Provides a system using multi-scale spatial weighting and interpolation to enhance image super-resolution. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/FMM.py))
- [Variance-Based Spatial Modulators](https://awesome-repositories.com/f/artificial-intelligence-ml/model-training/spatial-control-networks/spatially-adaptive-normalization/variance-based-spatial-modulators.md) — Implements a process using adaptive pooling and variance-based modulation to emphasize relevant spatial information. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(ECCV2024)SMFA.py))
- [Additive Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/multi-head-attention-mechanisms/additive-attention.md) — Provides an efficient attention mechanism combining query-weighted sums with adaptive average pooling. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/MHIASA.py))
- [Axis-Based Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/multi-head-attention-mechanisms/axis-based-attention.md) — Uses grouped Hadamard products and depth-wise convolutions across multiple axes to capture spatial and channel dependencies. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/GHPA.py))
- [Cross-Region Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/multi-head-attention-mechanisms/cross-region-attention.md) — Implements a self-attention mechanism across partitioned image regions using multi-head processing. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/CRMSA.py))
- [Cross-Scale Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/multi-head-attention-mechanisms/cross-scale-attention.md) — Captures dependencies across different vision-based scales using a multi-head self-attention mechanism. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(ICLR%202022)%20Crossformer.py))
- [Sparse Attention Routing](https://awesome-repositories.com/f/artificial-intelligence-ml/multi-head-attention-mechanisms/sparse-attention-routing.md) — Provides a gating mechanism that routes input tokens to a sparse subset of attention heads. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(arXiv%202024)%20MoHAttention.py))
- [Window-Based Attention](https://awesome-repositories.com/f/artificial-intelligence-ml/multi-head-attention-mechanisms/window-based-attention.md) — Provides a window-based multi-head self-attention mechanism with relative position bias for spatial relationship capture. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(ICPR%202022)%20MOATransformer.py))
- [Positional Embedding Layers](https://awesome-repositories.com/f/artificial-intelligence-ml/positional-encoding-techniques/positional-embedding-layers.md) — Creates sine and cosine positional encodings to provide spatial or sequential awareness to network layers. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/FCHiLo.py))

### Part of an Awesome List

- [Global Frequency Filters](https://awesome-repositories.com/f/awesome-lists/devtools/feature-extraction/frequency-domain/global-frequency-filters.md) — Provides a process that filters image features in the frequency domain using 2D Real Fast Fourier Transforms. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(NeurIPS%202021)%20GFNet.py))
- [Spectral Feature Filters](https://awesome-repositories.com/f/awesome-lists/devtools/feature-extraction/frequency-domain/spectral-feature-filters.md) — Implements frequency-domain transformations on images and time-series data to extract spectral features and apply dynamic filtering. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/tree/main/%E9%A2%91%E5%9F%9F))
- [Multi-Scale Self-Attention](https://awesome-repositories.com/f/awesome-lists/ai/attention-mechanisms/self-attention-implementations/multi-scale-self-attention.md) — Provides a polarized multi-scale feature self-attention mechanism to refine features across spatial scales. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/GlobalPMFSBlock.py))
- [Polarized Self-Attention](https://awesome-repositories.com/f/awesome-lists/ai/attention-mechanisms/self-attention-implementations/polarized-self-attention.md) — Computes channel-wise and spatial-wise attention weights for high-quality pixel-wise regression. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(arXiv%202021)%20PSAN.py))
- [Separable Self-Attention](https://awesome-repositories.com/f/awesome-lists/ai/attention-mechanisms/self-attention-implementations/separable-self-attention.md) — Implements a computationally efficient separable self-attention mechanism that condenses input sequences into context vectors. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/(arXiv%202021)%20MobileViTv2.py))
- [Transposed Self-Attention](https://awesome-repositories.com/f/awesome-lists/ai/attention-mechanisms/self-attention-implementations/transposed-self-attention.md) — Provides a transposed self-attention mechanism that reduces complexity by computing attention across channel dimensions. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/MDTA.py))

### Scientific & Mathematical Computing

- [Signal Processing](https://awesome-repositories.com/f/scientific-mathematical-computing/data-modeling-processing/signal-processing.md) — Implements discrete wavelet transforms and frequency-domain filtering for signal analysis within neural networks.
- [Wavelet Signal Decompositions](https://awesome-repositories.com/f/scientific-mathematical-computing/wavelet-signal-decompositions.md) — Implements discrete wavelet transforms to split signals into high and low frequency components for spatial analysis.
- [Inverse Wavelet Reconstruction](https://awesome-repositories.com/f/scientific-mathematical-computing/wavelet-signal-decompositions/inverse-wavelet-reconstruction.md) — Implements a process that restores original signals from decomposed wavelet components using inverse discrete wavelet transforms. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/blob/main/HWAB.py))
- [Neural Signal Processing Layers](https://awesome-repositories.com/f/scientific-mathematical-computing/wavelet-signal-decompositions/neural-signal-processing-layers.md) — Implements neural network modules for wavelet transforms and frequency-domain filtering for signal analysis.
- [Wavelet-Based Downsampling](https://awesome-repositories.com/f/scientific-mathematical-computing/wavelet-signal-decompositions/wavelet-based-downsampling.md) — Provides a process that reduces the sampling rate of input data using wavelet transforms to extract essential features. ([source](https://github.com/ai-dawang/PlugNPlay-Modules/tree/main/%E9%87%87%E6%A0%B7))

### Graphics & Multimedia

- [Image Feature Engineering](https://awesome-repositories.com/f/graphics-multimedia/image-feature-engineering.md) — Extracts and refines spatial and spectral representations using wavelets and Fourier transforms.