meta-llama/llama-stack

Features

• LLM Orchestrators - Implements a standardized orchestration layer that manages workflows between diverse LLM deployments and external tools via a unified API.
• AI Provider Gateways - Acts as a unified communication layer that routes requests to various AI model providers.
• Agentic LLM Frameworks - Provides a platform for building autonomous agents with integrated support for tool use, memory, and skill bundles.
• Agentic Tool Orchestration - Manages the discovery, planning, and execution of tool calls within autonomous agent workflows.
• AI Agent Orchestration - Coordinates specialized agents by combining custom instructions, tool execution, and document retrieval.
• AI Model APIs - Provides unified interfaces for interacting with diverse AI models through a single compatible API.
• LLM Provider Interfaces - Provides interfaces for communicating with various LLM providers to exchange messages and structured responses.
• Provider-Agnostic Model Interfaces - Provides an abstraction layer that standardizes inputs and outputs across multiple LLM providers.
• Tool Execution Orchestrators - Coordinates sequences of external server calls and file searches to automate complex, multi-step agent tasks.
• Vector Store Orchestrators - Ships a standardized interface to manage indexing and retrieval logic across vector stores to enable retrieval augmented generation.
• Unified Model Interfaces - Implements a standardized execution interface for processing and streaming across different language model providers.
• High Volume AI Processing - Manages large-scale asynchronous batch requests to optimize the processing of generative AI tasks.
• Standardized Model Communication Protocols - Implements a standardized messaging protocol to ensure seamless interoperability between diverse generative AI deployments.
• Vector Stores - Provides vector storage capabilities to index and search documents for retrieval-augmented generation.
• Vector-Store Augmented Generation - Uses vector databases to inject relevant document shards into model prompts for augmented generation.
• Skill Versioning Systems - Organizes agent capabilities using versioned manifest archives to ensure consistent function invocation.
• GenAI Application Deployment - Deployments of generative AI applications with integrated networking and vector database support.
• AI Stack Deployments - Provides a standardized software stack for deploying and managing large language model interfaces.
• Batch Request Processing - Implements an offline queue system to process large volumes of asynchronous requests for improved throughput and reduced costs.
• Skill Manifests - Uses declarative manifest files to define and version the tools and functions an agent can execute.
• Asynchronous Batch Requesting - Handles large volumes of generative AI requests through offline processing to increase throughput and reduce costs.
• Agent Frameworks - Core building blocks for deploying generative AI applications at scale.
• Application Frameworks - Standardized framework for building applications with Llama models.

Star history