5 repositorios
Mechanisms for sharing thread pools and memory buffers across concurrent model execution sessions.
Distinct from Shared Memory Buffers: Distinct from Shared Memory Buffers: focuses on session-level resource sharing for concurrent model execution rather than inter-process data passing.
Explore 5 awesome GitHub repositories matching data & databases · Runtime Resource Sharing. Refine with filters or upvote what's useful.
MNN is a high-performance inference engine and framework designed for on-device machine learning. It provides a comprehensive environment for executing, optimizing, and deploying neural network models directly on mobile and resource-constrained edge devices. The framework distinguishes itself through a robust model optimization toolkit that supports quantization, compression, and structural graph manipulation to minimize memory footprint and maximize execution speed. It features a modular architecture that abstracts hardware-specific backends, allowing models to run efficiently across diverse
Minimizes resource overhead by sharing thread pools and memory buffers across multiple concurrent model execution sessions.
RealtimeSTT is a local speech-to-text engine and real-time automatic speech recognition server. It utilizes transformer-based recognition and omnilingual pipelines to convert live audio streams into text, providing a WebSocket-based streaming API for raw PCM audio transmission. The project is distinguished by a dual-backend transcription pipeline that uses a lightweight engine for immediate partial suggestions and a heavier model for final high-accuracy results. It includes a wake word detection system to trigger recording and employs a shared-resource inference model to distribute heavy spee
Loads heavy speech models into memory once and shares them across multiple concurrent user sessions to minimize overhead.
Agency Swarm is a multi-agent orchestration framework and development kit designed to coordinate specialized AI agents through defined communication patterns and handoffs. It functions as a system for managing agent swarms, providing an API gateway to expose these coordinated collectives as production-ready HTTP endpoints. The project distinguishes itself through its Model Context Protocol integration layer, which connects agents to external data sources and capabilities. It implements specialized orchestration patterns, such as the orchestrator-worker model and role-based delegation, to tran
Distributes common instructions, tools, and files across a group of agents to ensure consistency.
Agent-OS is an LLM multi-agent orchestration framework and AI software development lifecycle tool designed to coordinate specialized agents through shared workspaces and structured task lists. It functions as an agentic application bootstrapper and technical specification engine, providing the infrastructure to guide the process from product requirements to automated coding and deployment. The system distinguishes itself through spec-driven development, using detailed technical specifications and layered context injection to ensure generated code aligns with project standards. It employs a ma
Provides a common environment where specialized agents share files, instructions, and task lists.
Oh-my-agent es un framework de orquestación agnóstico al proveedor diseñado para gestionar equipos de agentes autónomos y automatizar flujos de trabajo de ingeniería complejos. Funciona como una herramienta de desarrollo multi-agente que sincroniza el comportamiento, las habilidades y las reglas específicas del proyecto de los agentes en diversos entornos de desarrollo e interfaces de línea de comandos. La plataforma se distingue por la proyección basada en configuración, que mantiene una única fuente de verdad para las definiciones de agentes que se mapean en varios formatos de runtime específicos del proveedor. Al utilizar puentes de enlaces simbólicos multiplataforma y un registro de habilidades agnóstico al proveedor, asegura que las capacidades modulares y reutilizables permanezcan consistentes independientemente del asistente de codificación de IA o IDE que se esté utilizando. El sistema proporciona un conjunto completo de herramientas para gestionar el ciclo de vida del agente, incluyendo indexación semántica para la navegación de código, guardrails de ejecución con recursos limitados para gestionar el consumo de tokens y puertas de calidad automatizadas para seguridad y cumplimiento. Admite la orquestación de tareas de varios pasos a través de disparadores basados en intenciones, permitiendo la programación de trabajos de mantenimiento y la ejecución de binarios externos dentro de flujos de trabajo definidos. La configuración se gestiona a través de perfiles centralizados y sincronización automatizada, asegurando la integridad en los entornos de proyecto. El sistema está diseñado para ser instalado e inicializado como una capa fundamental para automatizar tareas de desarrollo, investigación e infraestructura dentro de un repositorio.
Distributes shared instructions and toolsets across agent teams to ensure behavioral consistency.