Projects sharing features with JARVIS

Auto-GPT is an autonomous agent framework designed for creating and deploying AI agents that use large language models to plan and execute complex goals independently. The system provides a comprehensive environment for managing the entire agent lifecycle, from initial design and testing to live production deployment. The project features a low-code workflow designer that allows users to define agent behaviors by connecting functional blocks in a visual interface. It includes an agent marketplace for discovering and deploying pre-configured agent templates and a standardized evaluation tool t

llm-universe is a structured learning resource and technical guide focused on the development of large language model applications. It serves as a curriculum for mastering model orchestration, the creation of autonomous conversational agents, and the implementation of retrieval-augmented generation systems. The project provides detailed instructions on connecting model APIs with memory and tools to create execution chains. It specifically covers the construction of retrieval pipelines, including the process of cleaning raw documents, generating embeddings, and integrating vector databases to

AudioGPT is an LLM-driven audio framework and processing suite that uses large language models to orchestrate neural audio pipelines. It functions as a multimodal audio generator and processing system, integrating a collection of pretrained models to handle speech synthesis, sound generation, and audio manipulation. The system is distinguished by its ability to generate audio from diverse inputs, including text and images, and its capacity to produce synchronized talking head videos. It also operates as a neural speech translator, converting spoken language between different tongues while pre

This project is a comprehensive machine learning interview guide and technical study resource designed for individuals preparing for machine learning and AI engineering roles. It provides a collection of materials and practice problems covering core algorithms, theoretical fundamentals, and the implementation of neural network architectures. The resource serves as a technical reference for generative AI development, focusing on the design and optimization of large language models and diffusion systems. It includes frameworks for system design, covering the architecture of production machine l

LangChain is a framework for building applications that chain large language models with external data sources and third-party tools. It serves as an orchestrator for autonomous agents that use language models to plan and execute multi-step tasks, while providing a toolkit for linking interoperable AI components into sequences to prototype complex model behaviors. The project provides a model agnostic integration layer, allowing users to switch between different language model providers using a standardized interface. It also includes tools for observability and evaluation to track the perfor

DSPy is a declarative programming framework designed for building complex language model applications. It treats model interactions as modular, composable programs, allowing developers to define task logic through typed class schemas rather than relying on manually written prompts. By organizing workflows into hierarchical, reusable Python objects, the framework enables the construction of sophisticated AI systems that manage state and execution flow independently. The framework distinguishes itself through an automated optimization engine that iteratively refines prompt instructions and few-

This project is an AI-powered IDE extension and LLM coding assistant that provides a conversational interface for generating, refactoring, and debugging code. It functions as an AI agent framework and a Model Context Protocol client, connecting AI models to external data sources and tools to automate complex development tasks. The system is distinguished by its use of autonomous AI agents capable of multi-step task execution, including the ability to read files, modify code, and run terminal commands iteratively. It supports recursive agent orchestration through subagent delegation and employ

Langflow is a low-code platform for designing and deploying multi-step AI agent pipelines and large language model sequences. It provides a visual environment to map logic and data flow between components, serving as an orchestrator for managing conversations and data retrieval across multiple autonomous agents. The platform distinguishes itself through a drag-and-drop interface that allows for the construction of complex AI pipelines without extensive boilerplate code. It enables the conversion of these internal workflows into standardized tools for external connectivity via the Model Contex

ComfyUI is a node-based generative AI orchestration engine designed for constructing, testing, and executing complex image and video synthesis pipelines. By utilizing a directed acyclic graph execution model, the platform allows users to build reproducible workflows through modular, interconnected processing blocks without requiring manual code implementation. It serves as both a local environment for high-performance model inference and a production-ready server for deploying generative capabilities. The platform distinguishes itself through its focus on workflow portability and extensibilit

Official repo for MM-REACT

[Website](http://craftjarvis-jarvis1.github.io/) [Paper](https://arxiv.org/abs/2311.05997) [Twitter](https://twitter.com/jeasinema/status/1723900032653643796)

GPT4Tools is an intelligent system that can automatically decide, control, and utilize different visual foundation models, allowing the user to interact with images during a conversation.

Note: https://github.com/kyegomez/tree-of-thoughts CANNOT replicate paper results.

ToolBench is an open platform for training, serving, and evaluating large language models that retrieve and call real-world APIs to complete user instructions. It provides an API-aware inference engine that selects relevant tools from a large corpus and generates sequences of tool calls to produce final answers, along with a custom API registration system that lets users add their own REST endpoints for the model to discover and invoke. The platform includes a complete instruction-tuning pipeline for training models on curated tool-use data, a multi-tool execution engine that coordinates sequ

This project aims to teach large language models (LLMs) to use external tools, enabling them as agents to for practical task planning and tool calling. To achieve this, we propose the "Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculum". Here is the brief…

Claude-flow is an autonomous agent coordination platform and orchestration framework designed for building complex, multi-step workflows powered by large language models. It functions as a TypeScript-based engine that decomposes high-level objectives into executable action sequences, enabling the creation of collaborative agent teams that operate with minimal manual oversight. The platform distinguishes itself through its ability to federate autonomous agents across network boundaries using secure communication channels and identity verification. It integrates a goal-oriented planning engine

AgenticSeek is a multi-agent orchestration system designed to decompose complex user objectives into granular, actionable tasks. By coordinating a team of specialized autonomous workers, the platform manages end-to-end workflows, ensuring that each component of a project is assigned to the most capable agent for execution. The system operates as a local-first runtime, executing all artificial intelligence models directly on user hardware to maintain data sovereignty and privacy. It integrates a browser automation engine for autonomous web research and interaction, alongside a sandboxed enviro

Llama-stack is a standardized orchestration stack and generative AI API gateway. It provides a unified communication layer and a consistent interface for deploying, managing, and interacting with various large language model providers and deployments. The system functions as an agent framework that manages tool execution and versioned skill bundles to automate complex tasks. It includes a batch processing system for handling large volumes of asynchronous requests through offline processing and a vector database interface for storing and searching documents to enable retrieval augmented genera

This project is a structured educational resource and technical guide for designing and implementing autonomous systems using large language models. It provides a comprehensive curriculum and code samples focused on agentic design patterns, autonomous development, and the creation of systems capable of planning and executing multi-step tasks. The resource details the implementation of agentic retrieval-augmented generation, where models autonomously plan and refine data searches. It covers a wide array of orchestrators and design patterns, including metacognitive reflection for self-correctin

OpenCompass is a comprehensive evaluation platform, benchmarking suite, and distributed model evaluator designed to measure the performance and accuracy of large language models. It provides a framework for benchmarking both open-source and API-based models against diverse datasets using standardized metrics and reproducible pipelines. The project features an automated judging framework that uses language models as judges to score and verify the quality of generated text. It includes a performance leaderboard system for comparing the relative capabilities of various models across industry-sta

Arize Phoenix is an LLM observability platform and evaluation framework designed to capture execution traces and monitor large language model applications. It serves as a prompt management system for versioning and testing templates, and as a self-hosted AI operations infrastructure for managing telemetry and experiments. The platform differentiates itself through a specialized embedding visualization tool used to detect data drift and optimize vector search. It provides a comprehensive evaluation suite that utilizes judge-based evaluators and ground-truth datasets to score model outputs, and

LangChain is an orchestration framework designed for building, managing, and deploying applications powered by large language models. It provides a unified integration layer that normalizes disparate model provider APIs into a consistent set of primitives, enabling developers to build complex, multi-step AI workflows that manage state, memory, and tool execution. The project distinguishes itself through a durable execution runtime that maintains persistent state across long-running processes by checkpointing progress to external storage. It models agent workflows as directed graphs, allowing

Accomplish is an artificial intelligence action framework and desktop automation agent designed to execute productivity tasks through natural language prompts. It functions as a workflow orchestrator that manages connections between various cloud and local language model providers to perform cross-platform operations. The system distinguishes itself through the ability to define and save stateful, reusable custom skills for recurring workflows. It integrates local application programming interfaces with third-party services to synchronize data and manage information across different platforms

Leon is a framework for building personal AI assistants that integrates large language models with local tool execution and persistent memory. It functions as an agentic workflow orchestrator and modular skill engine, enabling the creation of autonomous assistants capable of planning and executing multi-step tasks. The system features a retrieval-augmented generation memory architecture that indexes conversation history and user facts for context-aware grounding. It utilizes a modular skill system to interact with external binaries and APIs, supported by a loop that handles tool calling, sche

Claude Code is a command-line interface and multi-agent orchestration framework designed for autonomous software engineering. It enables AI agents to perform codebase modifications, debugging, and Git workflow management while coordinating multiple specialized agents to decompose and execute complex engineering tasks in parallel. The system distinguishes itself through a high degree of isolation and safety, utilizing Git worktrees to create independent working directories for concurrent agents and implementing a tiered permission system that combines user rules, project policies, and OS-level

UFO is a multi-device task orchestrator and LLM agent orchestration framework designed to decompose natural language requests into executable task graphs. It functions as a cross-platform UI automation tool capable of performing interactions on Windows and mobile devices while routing tasks to distributed agents based on their hardware and software capabilities. The system is distinguished by its RAG-enhanced agent architecture, which integrates external documentation and previous execution traces to improve decision-making. It employs a hybrid UI detection approach that combines computer vis