2 repository-uri
Awesome GitHub RepositoriesDistributed Block Switching Optimization
Optimizing the frequency of switching hosted model blocks to balance network traffic and compute throughput.
Distinguishing note: The candidates refer to kill-switches or UI toggles, not the operational logic of swapping model layers in a swarm.
Explore 2 awesome GitHub repositories matching software engineering & architecture · Distributed Block Switching Optimization. Refine with filters or upvote what's useful.
Awesome Distributed Block Switching Optimization GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- bigscience-workshop/petals
bigscience-workshop/petals
10,208Vezi pe GitHubPetals is a decentralized framework and inference engine for running large language models across a peer-to-peer network. It enables the execution of models that exceed the memory of any single machine by splitting computations and model layers across a collaborative swarm of GPUs. The system functions as a collaborative compute network where participants share local GPU resources and host model weights. It supports distributed prompt-tuning to adapt massive models to specific tasks and allows for the establishment of private compute swarms to process sensitive data within restricted, trusted
Provides mechanisms to adjust the frequency of hosted block switches to balance network traffic against swarm throughput.
Python - lostruins/koboldcpp
LostRuins/koboldcpp
9,511Vezi pe GitHubKoboldCPP is a local large language model inference engine and GGUF model runner designed to execute quantized models on personal hardware. It functions as a multimodal AI server and API gateway, providing OpenAI-compatible endpoints that allow third-party clients to interact with locally hosted models. The project distinguishes itself as an AI storytelling backend, featuring dedicated tools for long-form narrative management through persistent memory, world lore tracking, and character state management. It further extends its capabilities as a multimodal server capable of processing text, im
Saves cache snapshots to memory to avoid redundant computations when switching between different conversation contexts.
C++gemmaggmlgguf
