# xmu-xiaoma666/external-attention-pytorch

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/xmu-xiaoma666-external-attention-pytorch).**

12,176 stars · 1,944 forks · Python · MIT

## Links

- GitHub: https://github.com/xmu-xiaoma666/External-Attention-pytorch
- awesome-repositories: https://awesome-repositories.com/repository/xmu-xiaoma666-external-attention-pytorch.md

## Topics

`attention` `cbam` `excitation-networks` `linear-layers` `paper` `pytorch` `squeeze` `visual-tasks`

## Description

This is a PyTorch attention mechanism library and a collection of deep learning model components. It provides reference implementations of research-based attention mechanisms and neural network layers used to verify and understand deep learning papers.

The project facilitates deep learning research implementation and attention mechanism prototyping to capture global and local dependencies within complex datasets. It includes tools for neural network architecture design, specifically for building custom model components.

The library covers the development of multi-layer perceptrons, convolution blocks, and re-parameterized layers. These modular components are designed for PyTorch model development, allowing for the construction of reusable deep learning architectures.

## Tags

### Artificial Intelligence & ML

- [Attention Mechanisms](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/language-tools/tokenization-interfaces/tokenizer-base-interfaces/confidence-based-weighting/attention-mechanisms.md) — Provides tensor-based computations for attention mechanisms to capture global and local dependencies within sequences.
- [Attention Mechanisms](https://awesome-repositories.com/f/artificial-intelligence-ml/attention-mechanisms.md) — Implements various research-standard attention mechanisms to capture complex dependencies within datasets. ([source](https://github.com/xmu-xiaoma666/external-attention-pytorch#readme))
- [Module Builders](https://awesome-repositories.com/f/artificial-intelligence-ml/convolutional-neural-networks/module-builders.md) — Ships reference implementations for building reusable multi-layer perceptron and convolution blocks. ([source](https://github.com/xmu-xiaoma666/external-attention-pytorch#readme))
- [Deep Learning Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/architectures/neural-network-components/deep-learning-implementations.md) — Provides manual implementations of neural network layers and attention mechanisms from research papers for model verification.
- [Neural Network Layers](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers.md) — Provides a set of pre-defined architectural building blocks including MLPs and convolution blocks.
- [Modular Architectures](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-network-architectures/modular-architectures.md) — Implements a modular design for neural network layers to simplify the construction of complex deep learning architectures.
- [Neural Network Layers](https://awesome-repositories.com/f/artificial-intelligence-ml/neural-network-layers.md) — Provides architectural building blocks like MLPs and convolution blocks for designing custom neural network components.
- [PyTorch Model Components](https://awesome-repositories.com/f/artificial-intelligence-ml/pytorch-model-components.md) — Provides a library of modular PyTorch components for developing and evaluating neural network architectures.
- [Convolutional Block Composers](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/convolution-layers/convolutional-block-composers.md) — Provides modular components for grouping convolutional layers with normalization and activation for structural spatial feature extraction.
- [Re-parameterization Layers](https://awesome-repositories.com/f/artificial-intelligence-ml/machine-learning/frameworks/model-construction/neural-network-layers/convolution-layers/layered-architectures/re-parameterization-layers.md) — Implements re-parameterization layer folding to reduce computational overhead during inference without sacrificing model capacity.
- [Multilayer Perceptrons](https://awesome-repositories.com/f/artificial-intelligence-ml/multilayer-perceptrons.md) — Provides implementations of multilayer perceptrons using stacked fully connected layers to transform high-dimensional features.
- [Paper-to-Code Implementations](https://awesome-repositories.com/f/artificial-intelligence-ml/research-papers/paper-to-code-implementations.md) — Translates academic research papers into functional PyTorch implementations of attention mechanisms and layers.