3 repository-uri
Awesome GitHub RepositoriesLLM Token Compression
Techniques to reduce the number of billed tokens in LLM requests through deduplication and minification.
Distinct from Visual Token Compression: Focuses on textual LLM prompt tokens and message deduplication rather than visual or security token compression.
Explore 3 awesome GitHub repositories matching data & databases · LLM Token Compression. Refine with filters or upvote what's useful.
Awesome LLM Token Compression GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- diegosouzapw/omniroute
diegosouzapw/OmniRoute
6,391Vezi pe GitHubOmniRoute is a unified LLM API gateway that connects multiple AI providers to a single endpoint. Its primary purpose is to simplify the integration of various AI models into tools and agents by translating different provider formats into a standardized API. The project distinguishes itself through a multi-strategy request routing system that optimizes for cost, speed, and availability, including automatic model fallbacks and a circuit-breaker resilience model to isolate provider failures. It employs a local-first security posture, using AES-256-GCM encryption to store API keys and conversatio
Reduces billed token usage through semantic pruning and deduplication of prompts to maximize the context window.
TypeScript - blockrunai/clawrouter
BlockRunAI/ClawRouter
3,020Vezi pe GitHubClawRouter is an AI model router and API gateway designed to classify query complexity and assign prompts to the most efficient model tier. It operates as a multi-model AI proxy that orchestrates traffic between various large language models and AI media generators through a unified interface. The project distinguishes itself by integrating a non-custodial micropayment processor using the x402 protocol. This allows for per-request API access and USDC settlement on Base and Solana chains, replacing static API keys with wallet-based authentication and real-time budget enforcement. The system c
Reduces billed token counts by deduplicating messages and minifying data before sending requests.
TypeScriptaiai-agentsanthropic - aiming-lab/simplemem
aiming-lab/SimpleMem
2,972Vezi pe GitHubSimpleMem is a persistent memory system for AI assistants designed to maintain context across different user chat sessions. It functions as a memory server and multimodal vector database that stores and retrieves information from text, images, audio, and video. The project features a context compression engine that distills interaction histories into compact units to reduce token consumption. It utilizes a distributed memory orchestrator and worker-thread parallel processing to reduce latency when querying large-scale dialogue datasets. The system implements a hybrid indexing approach combin
Reduces token consumption by compressing interaction histories into compact, non-redundant units.
Python
