1 مستودع
Awesome GitHub RepositoriesToken-Limited Batching
Packing variable-length sequences into micro-batches up to a per-GPU token limit to maximize throughput.
Distinct from Dynamic Inference Batching: Distinct from Dynamic Inference Batching: focuses on training batching with per-sample loss preservation, not inference batching.
Explore 1 awesome GitHub repository matching data & databases · Token-Limited Batching. Refine with filters or upvote what's useful.
Awesome Token-Limited Batching GitHub Repositories
اعثر على أفضل المستودعات باستخدام الذكاء الاصطناعي.سنبحث عن أفضل المستودعات المطابقة باستخدام الذكاء الاصطناعي.
- thudm/slime
THUDM/slime
4,259عرض على GitHubSLIME is a distributed reinforcement learning framework for large language model post-training that bridges Megatron training with SGLang inference servers. It orchestrates scalable RL loops across GPU clusters, decoupling training and inference into independent processes that communicate over HTTP and NCCL for independent scaling and fault tolerance. The system supports multi-agent reinforcement learning workflows with parallel agent instances, customizable rollout strategies, and personalized agent serving that improves models from prior conversations without disrupting API serving. The fra
Packs variable-length sequences into batches up to a token limit per GPU, preserving per-sample loss while maximizing throughput.
Python
