meta-llama/llama-stack
Llama Stack
Llama-stack est une pile d'orchestration standardisée et une passerelle API pour l'IA générative. Il fournit une couche de communication unifiée et une interface cohérente pour déployer, gérer et interagir avec divers fournisseurs et déploiements de grands modèles de langage (LLM).
Le système fonctionne comme un framework d'agents qui gère l'exécution des outils et des bundles de compétences versionnés pour automatiser des tâches complexes. Il inclut un système de traitement par lots pour gérer de grands volumes de requêtes asynchrones via un traitement hors ligne et une interface de base de données vectorielle pour stocker et rechercher des documents afin de permettre la génération augmentée par récupération (RAG).
La pile couvre des capacités de haut niveau, notamment l'orchestration d'agents IA, le déploiement de modèles et la standardisation des API de modèles pour permettre de basculer entre les fournisseurs sans réécrire le code de l'application.
Features
- LLM Orchestrators - Implements a standardized orchestration layer that manages workflows between diverse LLM deployments and external tools via a unified API.
- AI Provider Gateways - Acts as a unified communication layer that routes requests to various AI model providers.
- Agentic LLM Frameworks - Provides a platform for building autonomous agents with integrated support for tool use, memory, and skill bundles.
- Agentic Tool Orchestration - Manages the discovery, planning, and execution of tool calls within autonomous agent workflows.
- AI Agent Orchestration - Coordinates specialized agents by combining custom instructions, tool execution, and document retrieval.
- AI Model APIs - Provides unified interfaces for interacting with diverse AI models through a single compatible API.
- LLM Provider Interfaces - Provides interfaces for communicating with various LLM providers to exchange messages and structured responses.
- Provider-Agnostic Model Interfaces - Provides an abstraction layer that standardizes inputs and outputs across multiple LLM providers.
- Tool Execution Orchestrators - Coordinates sequences of external server calls and file searches to automate complex, multi-step agent tasks.
- Vector Store Orchestrators - Ships a standardized interface to manage indexing and retrieval logic across vector stores to enable retrieval augmented generation.
- Unified Model Interfaces - Implements a standardized execution interface for processing and streaming across different language model providers.
- High Volume AI Processing - Manages large-scale asynchronous batch requests to optimize the processing of generative AI tasks.
- Standardized Model Communication Protocols - Implements a standardized messaging protocol to ensure seamless interoperability between diverse generative AI deployments.
- Vector Stores - Provides vector storage capabilities to index and search documents for retrieval-augmented generation.
- Vector-Store Augmented Generation - Uses vector databases to inject relevant document shards into model prompts for augmented generation.
- Skill Versioning Systems - Organizes agent capabilities using versioned manifest archives to ensure consistent function invocation.
- GenAI Application Deployment - Deployments of generative AI applications with integrated networking and vector database support.
- AI Stack Deployments - Provides a standardized software stack for deploying and managing large language model interfaces.
- Batch Request Processing - Implements an offline queue system to process large volumes of asynchronous requests for improved throughput and reduced costs.
- Skill Manifests - Uses declarative manifest files to define and version the tools and functions an agent can execute.
- Asynchronous Batch Requesting - Handles large volumes of generative AI requests through offline processing to increase throughput and reduce costs.
- Agent Frameworks - Core building blocks for deploying generative AI applications at scale.
- Application Frameworks - Standardized framework for building applications with Llama models.
