microsoft/JARVIS
JARVIS
JARVIS is a system for large language model task orchestration, deployment management, and automation benchmarking. It utilizes a task orchestrator to decompose complex requests into actionable steps and coordinates various expert models to synthesize final responses.
The project includes an AI model deployment manager to handle the local deployment of expert models across different hardware scales. It further provides an AI workflow API consisting of web endpoints used to trigger automated task workflows and retrieve results from model selection stages.
The framework incorporates an automation benchmark and evaluation suite to measure the ability of models to automate complex tasks using standardized datasets.
Features
- Task Planning Systems - Decomposes complex requests into actionable steps and coordinates expert model calls through an LLM-based planning system.
- LLM Application Orchestration - Coordinates large language models to plan complex requests and synthesize responses from multiple expert models.
- Automation Capability Benchmarks - Evaluates the capability of large language models to automate complex tasks using standardized benchmarking datasets.
- LLM Orchestrators - Uses a large language model to plan complex requests and coordinate expert models for final response synthesis.
- Local Model Management - Deploys and manages sets of expert models on local hardware across various scales and footprints.
- Domain-Expert Routing - Routes specific sub-tasks to specialized machine learning models based on their domain expertise and performance metrics.
- Multi-Model Workflow Coordinators - Coordinates large language models to sequence expert model calls and synthesize results into a final response.
- Automation Success Metrics - Implements a benchmark suite to measure model success in automating complex machine learning tasks using standardized datasets.
- Local Model Deployment - Manages the local deployment of flexible model instances that scale according to available hardware resources.
- LLM Evaluation - Evaluates the ability of large language models to automate complex tasks using standardized datasets and metrics.
- AI Workflow Serving - Creates web endpoints to trigger AI workflows and retrieve intermediate results from model selection stages.
- Workflow API Endpoints - Provides web endpoints that expose AI pipelines as programmable services for external triggering and result retrieval.
- Response Synthesis Pipelines - Combines intermediate outputs from various expert models into a single coherent final response through a refinement process.
- RESTful Workflow APIs - Wraps AI planning and execution pipelines as RESTful web endpoints for external programmatic triggering.
- Agent Tool Learning - Enhances agent capabilities with concise tool instructions.
- In Context Learning - Solving complex AI tasks by coordinating multiple foundation models.
- LLM Frameworks - System connecting language models with the broader machine learning community.
- Perception Models - Solving AI tasks by coordinating multiple foundation models.
- Reasoning And Planning - Connects language models to external AI tools for task solving.
- Reasoning Frameworks - Solving complex AI tasks by coordinating multiple models.
- Tool Optimization - Enhancing agents with concise tool instructions.
