1 repositorio
Awesome GitHub RepositoriesToken-Limited Batching
Packing variable-length sequences into micro-batches up to a per-GPU token limit to maximize throughput.
Distinct from Dynamic Inference Batching: Distinct from Dynamic Inference Batching: focuses on training batching with per-sample loss preservation, not inference batching.
Explore 1 awesome GitHub repository matching data & databases · Token-Limited Batching. Refine with filters or upvote what's useful.
Awesome Token-Limited Batching GitHub Repositories
Encuentra los mejores repositorios con IA.Buscaremos los repositorios que mejor coincidan usando IA.
- thudm/slime
THUDM/slime
4,259Ver en GitHubSLIME is a distributed reinforcement learning framework for large language model post-training that bridges Megatron training with SGLang inference servers. It orchestrates scalable RL loops across GPU clusters, decoupling training and inference into independent processes that communicate over HTTP and NCCL for independent scaling and fault tolerance. The system supports multi-agent reinforcement learning workflows with parallel agent instances, customizable rollout strategies, and personalized agent serving that improves models from prior conversations without disrupting API serving. The fra
Packs variable-length sequences into batches up to a token limit per GPU, preserving per-sample loss while maximizing throughput.
Python
