Awesome GitHub RepositoriesRing Attention Distributors

Splits a long sequence into blocks and distributes them across devices in a ring topology, computing attention incrementally to handle sequences longer than a single device's memory.

Distinct from Long-Context Sequence Processors: Distinct from Long-Context Sequence Processors: focuses on the ring topology distribution across devices rather than general memory-efficient sequence management.

Explore 1 awesome GitHub repository matching data & databases · Ring Attention Distributors. Refine with filters or upvote what's useful.