crmne/ruby_llm

Features

• Autonomous Agent Orchestration - Functions as an orchestrator for building and deploying autonomous agents with persistent memory and complex workflows.
• Provider-Agnostic Model Interfaces - Standardizes diverse LLM provider APIs into a consistent interface to eliminate provider-specific parsing logic.
• AI Agent Definitions - Allows the creation of specialized assistants by combining models, instructions, and toolsets into reusable personas.
• AI Agent Orchestration - Provides a framework for coordinating specialized AI agents with custom personas, instructions, and toolsets.
• AI Code Interpreters - Implements environments where AI models generate and execute code to perform calculations or system interactions.
• AI Conversation Managers - Offers interfaces for managing multiple AI model sessions, including history and configuration parameters.
• AI Observability Suites - Provides a diagnostic suite for tracing, monitoring token usage, and calculating the costs of AI workflows.
• Persona and Behavioral Instructions - Defines AI personas and behavioral constraints using persistent system prompts throughout conversations.
• AI Provider Managers - Includes utilities for managing API keys and authentication connections across multiple AI services.
• AI Workflow Orchestrators - Coordinates complex task sequences using sequential, parallel, and routing execution patterns.
• Language Model Integrations - Provides adapters and interfaces to send prompts and manage interactions with various hosted or local language model providers.
• Chat History APIs - Provides interfaces for persisting and querying conversational message logs and history in a database.
• Chat State Management - Provides headless management of chat history and streaming responses integrated into database models.
• Code Execution Tools - Provides built-in tools for agents to perform real-world calculations and logic via code execution.
• Runtime Context Injections - Passes session-specific data and user inputs to tools without exposing them to the LLM schema.
• Conversation History Management - Provides tools for storing and retrieving logs of agent-user interactions to maintain context.
• Database Model Integrations - Connects chat capabilities directly to database models for easy persistence and retrieval.
• External Tool Execution - Triggers local methods to perform actions or fetch real-time data based on model requests.
• Function Calling Interfaces - Enables language models to execute external tools and API functions to extend their capabilities.
• Image Generation - Creates images from text prompts using a unified interface that abstracts different provider requirements.
• Image Generation Models - Generates original visual content from text descriptions using AI painting models.
• Persistent Chat Histories - Maintains long-term conversational context by saving chat sessions and messages to a database.
• Conversational Response Generation - Produces conversational chat responses through a unified interface connecting to multiple LLM providers.
• LLM Application Infrastructure - Provides infrastructure for managing model registries, API keys, and a provider-agnostic interface.
• LLM Integration Frameworks - Serves as a unified integration framework for connecting applications to various large language model providers.
• LLM Response Streaming - Delivers language model outputs incrementally as standardized chunks to reduce perceived latency.
• Multi-Modal Input Processors - Implements a unified interface to ingest and normalize images, audio, videos, and PDFs for model processing.
• Tool Call Executions - Executes custom Ruby classes as tools and persists the resulting interactions as conversation messages.
• Model Routing Registries - Maps model identifiers to routing logic and pricing metadata via a structured database registry.
• Model Provider Abstractions - Provides a unified interface that normalizes API interactions across multiple AI service providers.
• Multimodal AI Toolkits - Provides a comprehensive toolkit for processing text, images, audio, and video using a consistent API.
• Multimodal Document Processing - Processes images, videos, audio, and documents to extract information and summaries through a unified interface.
• Multimodal Integration Libraries - Integrates diverse media types including images, audio, video, and PDFs into AI pipelines via a unified API.
• Structured Output Generators - Forces language models to produce strictly typed, machine-readable JSON data formats for programmatic parsing.
• Tool Schema Definitions - Creates executable functions with structured schemas that models use to interact with external systems.
• RAG Context Retrieval - Implements retrieval-augmented generation to provide relevant grounded context from knowledge bases to the model.
• Structured Data Extraction - Enforces and extracts structured JSON formats from model outputs for predictable programmatic parsing.
• Structured Output Enforcements - Enforces JSON response formats using domain-specific language definitions to ensure predictable programmatic parsing.
• Text Embeddings - Transforms text strings into numerical vector representations to enable semantic search and similarity analysis.
• Tool Calling Implementations - Connects local code functions to a model to perform real-world actions or retrieve data.
• Dynamic Tool Schema Injection - Dynamically injects structured tool schemas into model prompts to enable real-time interaction with local Ruby methods.
• Vector Embeddings - Transforms raw text into numerical vector representations for use in semantic search and retrieval.
• Visual Input Processing - Analyzes visual content from images and PDF documents using large language models to extract information.
• AI Agents and Assistants - Provides a framework for building specialized AI assistants with custom personas and complex workflows.
• Agent State Persistence - Provides mechanisms to persist agent sessions, chat history, and internal context directly within database models.
• Conversation History Stores - Stores chats, messages, and system instructions in a database to maintain state across user sessions.
• AI - Integrates chat and message capabilities into database models to handle persistence and associations.
• Structured Data Schemas - Creates JSON schemas using a domain-specific language to enforce structured data output from models.
• Tool Interaction Streams - Yields partial text and tool requests while resuming output delivery after tool execution completes.
• Event Monitoring Systems - Broadcasts AI-related internal events and task lifecycle data for monitoring and observability.
• AI Observability - Offers specialized tracking for token usage, cost calculation, and workflow tracing via OpenTelemetry.
• Token Usage Analytics - Calculates normalized input, output, and cached token counts for accurate usage tracking and billing.
• Cost and Token Trackers - Estimates financial costs of AI interactions by applying model-specific pricing registries to token usage.
• Token Cost Calculators - Computes financial usage expenses by applying model-specific pricing to normalized token counts.
• AI Response Adapters - Uses adapters to standardize diverse AI model API responses into a consistent internal object format.
• AI Workflow Pipelines - Coordinates multi-step AI task sequences using a pipeline of evaluators and optimizers to refine outputs.
• Agent File Attachments - Provides mechanisms to attach files and documents to messages to enable multimodal context for AI agents.
• Parallel Tool Execution - Runs multiple tool calls in parallel using lightweight threads to reduce overall agent latency.
• Model Registries - Provides a database-backed catalogue of available models and their provider routing configurations.
• Multi-Provider Chat Interfaces - Provides controllers and views for building real-time chat interfaces that support multiple AI providers.
• AI Performance Monitoring - Tracks AI operation efficiency, throughput, and response times via a monitoring dashboard with automated alerts.
• Audio Transcriptions - Implements capabilities to convert spoken audio recordings into written text for analysis.
• Automated Output Evaluation - Provides automated quality and safety checks for model responses using deterministic assertions and secondary model evaluations.
• Evaluator-Optimizer Loops - Implements an iterative loop between an evaluator and an optimizer to refine the quality of AI outputs.
• Prompt Configuration Files - Supports loading system instructions from external template files to separate prompt engineering from application code.
• Local Model Execution - Executes quantized models within the local process to enable offline capabilities and ensure data privacy.
• Model Capability Registries - Provides a registry to track available models and automatically detect their capabilities and pricing.
• Model Metadata Registrars - Implements interfaces for defining and persisting configuration metadata for external AI models.
• Model Context Protocol Integrations - Integrates the standardized Model Context Protocol to link AI models to external tools and resources.
• Model Identifier Mappings - Resolves user-defined model aliases and unique identifiers to specific model instances and backends.
• Real-Time Audio Transcribers - Provides a unified interface to convert spoken language from various audio file formats into text.
• Reasoning Capture Utilities - Captures and extracts the internal thinking process of reasoning-capable models to provide visibility into their logic.
• Reasoning Process Controllers - Manages the reasoning process by persisting internal thinking steps before delivering a final answer.
• Unified Audio Transcription - Converts spoken language from audio files into written text.
• Multimodal Context Injection - Enables sending files or documents back to the model alongside text to facilitate visual analysis.
• AI Agent Deployment Frameworks - Launches reusable assistants capable of performing complex tasks autonomously within production environments.
• Agent Execution Traces - Sends OpenTelemetry traces of agent decisions and tool calls to backends for execution flow analysis.
• Sensitive Data Redaction - Removes confidential data from conversations to prevent sensitive information from reaching external AI providers.
• Reasoning - Saves reasoning output and token counts to a database for auditing and cost tracking.
• Document and Unstructured Extraction - Parses information from files like PDFs and CSVs to convert unstructured content into usable data.
• Application Debug Logging - Logs detailed request and response data to troubleshoot communication between the application and AI providers.
• AI Audio Analysis - Provides capabilities to accept audio files or streams for transcription and voice interaction.
• Automated Media Analyzers - Extracts insights and data from images and videos using integrated computer vision processing.
• RPC Traffic Debugging - Provides diagnostic logging of raw message exchanges to troubleshoot AI provider integrations.
• Content Moderation - Analyzes model inputs and outputs using moderation models to identify and block harmful content.
• Fiber-Based Concurrent Execution - Utilizes lightweight fibers to execute multiple tool calls and AI requests in parallel without blocking the main process.
• Execution Telemetry Pipelines - Implements telemetry pipelines that broadcast internal execution data and token usage for monitoring and OpenTelemetry tracing.
• Concurrent AI Requesting - Dispatches multiple language model queries using non-blocking tasks to prevent application freezing.
• Model Interaction Monitors - Emits event metadata for model requests, completions, and images to monitor usage and provider performance.
• AI Provider Routing - Implements dynamic routing of requests across different AI model providers and API keys on a per-chat basis.

Star history