deepseek-ai/DeepEP

Features

• Distributed Training Accelerators - Provides a distributed acceleration system that overlaps communication with computation for large-scale neural networks.
• Communication-Computation Overlap - Coordinates concurrent GPU streams to overlap communication and computation, hiding network latency.
• Expert Parallel Communication Libraries - Implements a specialized library for high-throughput data exchange between GPUs to optimize MoE models.
• Expert Communication Kernels - Manages high-throughput GPU data exchange to facilitate dispatch and combine operations in MoE architectures.
• Expert Exchange Kernels - Provides specialized GPU communication kernels to handle high-throughput expert data exchange.
• Communication Kernels - Ships dynamically compiled GPU kernels for efficient data dispatch and combination in distributed environments.
• Dynamic Kernel Optimization - Dynamically generates specialized GPU kernels during execution to improve performance and eliminate manual compilation.
• Just-In-Time Kernel Compilers - Generates optimized GPU kernels during execution to remove the need for pre-installation compilation.
• Large-Scale Model Training - Optimizes the movement of large-scale data across multiple GPUs during distributed model training.
• Expert Data Exchange - Facilitates high-throughput data exchange between GPUs for the dispatch and combine phases of MoE models.
• Expert-Parallel Data Exchange - Moves large volumes of data across multiple GPUs to maintain performance during expert-parallel model training.
• Remote GPU Memory Coordination - Uses low-level primitives to access remote GPU memory and coordinate pipeline parallelism across nodes.
• Remote GPU Memory Interfaces - Provides a low-level remote GPU memory interface to coordinate pipeline and context parallelism.
• Remote GPU Memory Access - Implements low-level memory primitives for coordinating direct data movement across distributed GPUs.
• JIT Kernel Compilers - Implements JIT kernel compilation to generate hardware-optimized GPU kernels at runtime.
• Network Traffic Management - Implements network traffic management by assigning workloads to specific virtual lanes to prevent interference.
• Virtual-Lane Isolation - Uses virtual-lane traffic isolation and network service levels to prevent interference between data streams.
• Cache-Aware Memory Access - Utilizes low-level memory primitives to coordinate distributed parallelism and optimize hardware resource usage.
• Attention Optimization - Expert parallelism management for mixture-of-experts models.
• Distributed Training Tools - Communication library for expert-parallelism in MoE model training and inference.
• Computation and Optimization - Communication library for mixture-of-experts and expert parallelism.

Star history