awesome-repositories.com
Blog
awesome-repositories.com

Entdecke die besten Open-Source-Repositories mit KI-gestützter Suche.

EntdeckenKuratierte SuchenOpen-Source-AlternativenSelf-hosted SoftwareBlogSitemap
ProjektÜber unsRanking-MethodikPresseMCP-Server
RechtlichesDatenschutzAGB
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·
mlflow avatar

mlflow/mlflow

0
View on GitHub↗
mlflow.org↗

Mlflow

Features

  • Agent Evaluation Tools - Analyze agent performance by defining test datasets and custom scorers to assess both final outputs and intermediate tool usage.
  • AI Gateways - Manage access to external AI providers by configuring API keys and defining model endpoints through a web interface.
  • Experiment Tracking - Defines unique experiment names, specifies artifact storage locations, and attaches metadata tags to track development progress.
  • Experiment Tracking Servers - Start a local tracking server with a single command to manage experiments, model artifacts, and metadata for machine learning workflows.

KI-Suche

Entdecke weitere awesome Repositories

Beschreibe in einfachen Worten, was du brauchst — die KI bewertet tausende kuratierte Open-Source-Projekte nach Relevanz.

Start searching with AI
  • Experiment Tracking Systems - Log experiment parameters, metrics, and models during training using explicit API calls or automatic logging for popular training libraries.
  • LLM Execution Tracing - Captures detailed model call data including prompts, completions, and token counts.
  • Model Gateways - Aggregates multiple AI service providers behind a single interface to manage authentication, rate limiting, and cost tracking for model requests.
  • Model Lifecycle Management - Centralizes version control, tracks model lineage, and organizes deployment workflows through a structured registry system.
  • Model Registries - Tracks versions, assigns aliases, and manages metadata for production-ready deployment workflows.
  • AI Observability Suites - A diagnostic toolkit for tracing, evaluating, and monitoring the performance, quality, and operational costs of complex language model applications.
  • Custom Evaluation Judges - Adapt base models to act as custom judges that align with specific business requirements and expert human judgment.
  • Evaluation Frameworks - Configure the LLM used to power evaluation judges by specifying the provider and model name in the judge definition.
  • LLM Provider Integrations - Connect various language model providers into a unified gateway to access chat and embedding capabilities through a consistent interface.
  • Model Evaluation Frameworks - Runs systematic evaluations using built-in metrics to track quality and detect regressions in model performance.
  • Model Inference - Sends input payloads to a specified endpoint or deployment target to generate predictions from a deployed model.
  • Model Inference Servers - Deploy machine learning models to diverse environments by launching inference servers with REST endpoints for handling prediction requests.
  • Model Packaging - Organizes model artifacts and configuration files into a standardized directory format that supports multiple interoperable model flavors.
  • Model Serving Servers - Launch a local HTTP inference server using command-line tools to serve model predictions via standard REST endpoints.
  • Prompt Engineering Environments - Offers a collaborative environment for versioning, testing, and optimizing prompt templates with integrated lineage tracking and automated evaluation workflows.
  • Prompt Management Systems - Versions and deploys prompts with full lineage tracking while using automated algorithms to improve output quality.
  • Prompt Registries - Creates versioned prompt templates with metadata and response format specifications.
  • RAG Evaluation Frameworks - Evaluate RAG application performance using built-in judges that assess retrieval relevance, groundedness, and context sufficiency based on captured application traces.
  • Prompt Repositories - Maintains prompt templates with version control and A/B testing capabilities.
  • Agent Execution Tracing - Visualizes complex agent reasoning steps, tool calls, and retrieval processes for debugging.
  • Automatic Tracing Instrumentation - Enables comprehensive observability by capturing execution details with minimal code changes.
  • LLM Execution Tracing - Captures inputs, outputs, and execution details to provide full visibility into LLM behavior.
  • LLM Evaluation - Provides a dedicated environment for evaluating LLM application performance through prediction functions and datasets.
  • Model Serving - Launches a local webserver that accepts prediction requests in various data formats to serve saved models.
  • Agent Deployment Servers - Provides a dedicated server environment for deploying and managing autonomous AI agents with integrated request validation.
  • AI Gateway Management - Update existing endpoints by modifying model settings, connection configurations, or deleting them when they are no longer required.
  • AI Observability Tools - Captures end-to-end execution traces and performance metrics to debug complex agent workflows and monitor production behavior in real-time.
  • AI Observability Tracing - Captures complete execution traces of applications to monitor production quality, costs, and safety.
  • Artifact Logging - Associates a local file with a specific run by logging it as an artifact.
  • Automated Model Judges - Tracks quality metrics over time using automated judges to detect regressions and performance issues.
  • Automatic Logging - Captures model parameters, metrics, and artifacts automatically by calling a single function before executing training code.
  • Conversation Evaluation Tools - Evaluate existing conversation traces by grouping them via session IDs and applying multi-turn scorers to analyze production data.
  • Experiment Visualization Dashboards - Visualize and compare logged experiments, runs, and performance metrics through a web-based interface or a hosted tracking server.
  • Human Feedback Collection - Gather input from users and experts to validate judge accuracy, identify performance gaps, and improve overall evaluation quality.
  • LLM Tracing Systems - Instruments LLM applications to capture and send execution traces to a tracking server for observability.
  • Model Flavors - Ensures compatibility across various machine learning libraries and deployment tools by using a unified interface for loading and scoring models.
  • Model Serialization Formats - Standardizes models into portable directory formats to ensure consistent loading across environments.
  • Model Serving Endpoints - Deploys validated prompt configurations to live endpoints for real-time production inference.
  • Model Versioning - Log models with input and output signatures to define expected data formats, enabling better model understanding and validation for downstream deployment.
  • Observability Tools - Provides a visual interface to debug agent workflows and identify performance bottlenecks.
  • Experiment Tracking Tools - Start a local tracking server using the command line interface to manage experiments, models, and metadata for development workflows.
  • Distributed Trace Propagation - Maintains unified trace context across distributed services using W3C TraceContext headers.
  • Model Access Governance - Provides centralized governance for model access, including rate limiting and cost tracking.
  • AI Cost Monitoring - Tracks token usage and model efficiency to identify optimization opportunities.
  • Model Evaluation - Allows developers to define custom functions to measure specific quality aspects of AI applications.
  • Agent Deployment Tools - Launches agents using a server that provides automatic request validation and tracing for rapid production deployment.
  • Agent Simulation Environments - Simulate user interactions with an agent by defining test cases with specific goals and personas to generate and evaluate diverse conversation scenarios.
  • AI Application Monitoring - Captures complete traces of applications to monitor production quality, costs, and safety using standard observability tools.
  • Automated Instrumentation Frameworks - Instruments application logic automatically by integrating with agent frameworks to capture execution traces without manual code changes.
  • Conversational Evaluation Suites - Assess conversational agents by simulating multi-turn dialogues and applying scorers to evaluate interaction quality and safety at every step.
  • Custom Model Definitions - Defines custom inference logic and artifact dependencies to deploy models as standard functions.
  • Model Checkpoint Managers - Save multiple model checkpoints during a single training run and link performance metrics to specific versions for improved traceability.
  • Model Comparison Tools - Analyze evaluation results across different agent versions side-by-side to identify regressions, debug issues, and inform future development.
  • Model Containerization Tools - Package machine learning models into standardized containers with their dependencies and metadata to ensure consistent execution across various deployment environments.
  • Model Deployment Pipelines - Provides a standardized framework for packaging, versioning, and deploying machine learning models across diverse production environments and serving infrastructures.
  • Model Serialization - Save trained Keras models as artifacts and load them back for inference using dedicated functions that handle model serialization and retrieval.
  • Prompt Lifecycle Management - Manages prompt evolution and deployment pipelines using versioned templates and aliases.
  • Prompt Optimization Frameworks - Improves prompt performance automatically using genetic algorithms and metaprompting techniques.
  • Training Instrumentation - Integrate tracking calls directly into training loops to manually log custom metrics, hyperparameters, and model states during development.
  • LLM Development Frameworks - Platform for tracking experiments, evaluation, and model deployment.
  • Machine Learning - Integrated platform for end-to-end machine learning workflows.
  • Machine-Learning-Frameworks - Platform for managing the machine learning lifecycle.
  • Machine Learning Operations - Open-source platform for the complete machine learning lifecycle.
  • MLOps Platforms - Manages the end-to-end machine learning lifecycle.
  • Model Management - Platform for ML lifecycle management.
  • Observability and Evaluation - Comprehensive suite for LLM tracing and model evaluation.
  • Observability And Monitoring - End-to-end platform for tracking and monitoring model applications.
  • Perception and Machine Learning - Platform for managing the machine learning lifecycle.
  • Experiment and Data Management - Lifecycle management platform for experimentation and deployment.
  • Experimentation Tracking - Manages the end-to-end machine learning lifecycle.
  • Artifact Management - Transfers artifact files or directories from a remote repository to a local filesystem destination for further analysis.
  • AI Compliance Governance - Maintains audit trails and enforces content guardrails to meet organizational compliance requirements.
  • Application Quality Monitoring - Tracks quality metrics over time to proactively identify and resolve issues.
  • Automated Trace Evaluation - Configures automated judges to evaluate production traces for quality monitoring.
  • Dynamic Autologging - Automatically captures training metrics and artifacts by injecting tracking logic into libraries at runtime.
  • OpenTelemetry Exporters - Standardizes the transmission of execution traces by adhering to open observability protocols.
  • AI Auto-Logging Tools - Instruments AI applications with minimal effort by using built-in auto-logging capabilities for supported libraries.
  • AI Request Routers - Directs requests to various providers through a unified interface to manage rate limits and service fallbacks.
  • Autologging Controls - Toggles tracking functions to enable or disable automatic data collection for specific libraries.
  • Data Lineage - Track data sources automatically during model training by logging paths, formats, and versions of datasets read from distributed storage systems.
  • Data Lineage Trackers - Record metadata about training datasets to maintain lineage and traceability between data sources and final model performance.
  • Deep Learning Experiment Trackers - Capture training metrics, model parameters, and artifacts automatically during Keras model training by enabling a single-line configuration function.
  • Inference Clients - Interact with model servers using standard HTTP requests to perform inference, check health status, and retrieve version information.
  • Model Benchmarking Interfaces - Test registered models against defined datasets and scorers by wrapping them in a prediction function and passing them to the evaluation interface.
  • Model Configuration Management - Stores inference parameters with prompt templates to ensure consistent model behavior.
  • Model Signatures - Specifies input and output schemas with examples to enable automated validation and testing.
  • Model Validation Schemas - Enforces data integrity by validating runtime payloads against predefined signatures during model serving.
  • Transformer Model Management - Log transformer pipelines and individual model components to track model artifacts, metadata, and prompt templates for reproducible workflows.
  • ML Orchestration Deployments - Deploy the tracking server using container orchestration tools or managed cloud services for production-scale environments.
  • Authentication Clients - Supports multiple authentication methods for secure programmatic access to the tracking server.
  • Application State Versioning - Snapshots code and configurations into versioned entities for reliable reproduction.
  • Asynchronous Tracing - Captures execution context across asynchronous functions to ensure accurate tracing.
  • Workspace Management Systems - Organizes resources into isolated workspaces to share infrastructure across teams while maintaining strict boundaries.
  • Decorator-Based Instrumentation - Captures function inputs and outputs by wrapping code in decorators for performance analysis.
  • Execution Path Visualization - Visualizes complex execution paths in multi-step agents to make every step debuggable.
  • Performance Trend Analysis - Identifies performance patterns across large-scale deployments using summary interfaces.
  • Trace Querying - Provides a query language to filter traces based on attributes and tags.
  • Trace Sampling - Manages export volume using global sampling ratios and per-endpoint overrides.
  • Usage Monitoring Tools - Provides aggregated visibility into token usage and financial costs for model calls.
  • Execution Tracers - Enables deep inspection of agent execution traces to validate retrieval and tool usage.
  • Performance Profiling - Analyze logged experiment traces to discover operational bottlenecks and quality issues, generating a comprehensive report with actionable recommendations for improvement.
  • Autologging Customization - Passes arguments to initialization functions to customize model signature logging and refine data collection settings.
  • Batch Inference Tools - Process input data in bulk using command-line tools or scripts to generate predictions and save results to an output file.
  • Decorator-based Scorers - Create custom evaluation logic using a decorator to process application inputs, outputs, and execution traces for automated performance assessment.
  • Dependency Management - Infers required packages automatically and defines custom environment configurations to ensure consistent model execution.
  • Embedding Model Utilities - Log and load sentence transformer models with full metadata, model signatures, and support for native loading and generic inference interfaces.
  • Experiment Identifiers - Fetches the unique identifier for a newly created experiment after successfully submitting a creation request.
  • ML Infrastructure Configuration - Configure the tracking server architecture by plugging in custom backend stores for metadata and artifact stores for large files.
  • Model Signature Generators - Generate model signatures automatically using language-specific type hints to enable runtime data validation and improve development environment support.
  • OpenTelemetry Exporters - Exports OpenTelemetry traces from any language or framework by configuring the OTLP endpoint.
  • Prompt Caching Strategies - Configures memory-based caching to improve retrieval performance for prompt requests.
  • Semantic Search - Build semantic search systems by logging model parameters, saving corpus artifacts, and performing similarity searches using encoded document embeddings.
  • Stateful Evaluation Scorers - Implement complex evaluation logic by extending a base class and overriding the call method to create stateful scoring behaviors.
  • Access Control Policies - Controls access to resources by assigning granular permissions that define user interaction with assets.
  • Authentication Middleware - Secure the tracking server using built-in authentication methods and network protection middleware to prevent unauthorized access.
  • OIDC Authentication Plugins - Integrates with external identity providers to manage user sessions via OIDC.
  • Declarative Tracing - Captures inputs, outputs, and execution time using decorators that maintain call relationships.
  • Multi-tenant Isolation Policies - Enforces data and access boundaries by prefixing storage paths and applying granular permissions to separate assets across different organizational teams.
  • Telemetry Systems - Offloads trace and metric data collection to background processes to maintain application performance during high-throughput operations.
  • Tracing Instrumentation - Allows dynamic customization of span names and attributes during function execution.
  • Wrapper-based Instrumentation - Wraps existing functions to capture execution context without modifying original definitions.
  • Automated Trace Diagnostics - Automatically detects quality and operational issues within captured application traces.
  • Observability Controls - Provides global control over trace collection to manage observability overhead and privacy.
  • Programmatic Trace Analysis - Retrieves trace data via a Python API for analysis as structured data frames.
  • Resource Monitoring - Monitor hardware resource utilization, including GPU, CPU, disk, and network metrics, to identify performance bottlenecks and optimize training efficiency.
  • Session Tracking - Groups related execution traces into user sessions to analyze multi-turn conversation flows.
  • Trace Context Management - Attaches request and user metadata to production traces for improved debugging.
    • 26,554 Stars·5,852 Forks·Python·Apache-2.0·26 Aufrufe

    Star-Verlauf

    Star-Verlauf für mlflow/mlflowStar-Verlauf für mlflow/mlflow

    Open-Source-Alternativen zu Mlflow

    Ähnliche Open-Source-Projekte, sortiert nach der Anzahl der gemeinsamen Funktionen mit Mlflow.
    • agenta-ai/agentaAvatar von Agenta-AI

      Agenta-AI/agenta

      3,860Auf GitHub ansehen↗

      Agenta is a Prompt Ops lifecycle manager and prompt management platform that decouples prompt engineering from application code. It serves as a centralized system for developing, versioning, and deploying prompt templates and model configurations across different environments. The platform functions as an AI agent orchestrator with a visual interface for building agent workflows and connecting models to external tools. It further acts as an evaluation framework and observability tool, utilizing OpenTelemetry to capture execution traces, monitor latency, and track token costs. The system cove

      TypeScriptagentsevaluationllm-as-a-judge
      Auf GitHub ansehen↗3,860
    • langchain-ai/deepagentsAvatar von langchain-ai

      langchain-ai/deepagents

      25,006Auf GitHub ansehen↗

      Deepagents is an LLM agent orchestration platform and stateful application server designed for deploying and managing AI agents built with computational graphs. It provides a containerized runtime environment that handles agent execution, state persistence, and the versioning of AI assistants. The platform distinguishes itself through deep integration with the Model Context Protocol, allowing agents to function as servers that expose tools and capabilities to external clients. It features a sophisticated observability suite for capturing execution traces, performing LLM-based evaluations agai

      Pythonagentsdeepagentslangchain
      Auf GitHub ansehen↗25,006
    • portkey-ai/gatewayAvatar von Portkey-AI

      Portkey-AI/gateway

      12,091Auf GitHub ansehen↗

      This project is an artificial intelligence gateway that functions as a centralized middleware layer for managing, securing, and observing interactions with language, vision, and audio models. It provides a unified interface that standardizes requests across multiple providers, enabling teams to integrate AI capabilities into their applications through a consistent set of tools and protocols. The gateway distinguishes itself through its comprehensive infrastructure governance and traffic management capabilities. It allows for policy-driven routing, automated failover, and load balancing across

      TypeScriptai-gatewaygatewaygenerative-ai
      Auf GitHub ansehen↗12,091
    • aimhubio/aimAvatar von aimhubio

      aimhubio/aim

      6,159Auf GitHub ansehen↗

      Aim is an open-source platform for logging, visualizing, and comparing machine learning training runs and LLM traces. It provides a remote tracking server and a comparison UI, functioning as an ML experiment tracker, AI workflow logger, and LLM trace recorder that captures prompts, generations, and tool calls from AI applications. The platform distinguishes itself through a run-based data model with local SQLite storage, real-time metric streaming, and a plugin-based explorer system that supports specialized visual analysis of metrics, images, audio, and text. It offers a Python SDK with cont

      Python
      Auf GitHub ansehen↗6,159
    Alle 30 Alternativen zu Mlflow anzeigen→

    Häufig gestellte Fragen

    Was sind die Hauptfunktionen von mlflow/mlflow?

    Die Hauptfunktionen von mlflow/mlflow sind: Agent Evaluation Tools, AI Gateways, Experiment Tracking, Experiment Tracking Servers, Experiment Tracking Systems, LLM Execution Tracing, Model Gateways, Model Lifecycle Management.

    Welche Open-Source-Alternativen gibt es zu mlflow/mlflow?

    Open-Source-Alternativen zu mlflow/mlflow sind unter anderem: agenta-ai/agenta — Agenta is a Prompt Ops lifecycle manager and prompt management platform that decouples prompt engineering from… langchain-ai/deepagents — Deepagents is an LLM agent orchestration platform and stateful application server designed for deploying and managing… portkey-ai/gateway — This project is an artificial intelligence gateway that functions as a centralized middleware layer for managing,… aimhubio/aim — Aim is an open-source platform for logging, visualizing, and comparing machine learning training runs and LLM traces.… arize-ai/phoenix — Arize Phoenix is an LLM observability platform and evaluation framework designed to capture execution traces and… vibrantlabsai/ragas — Ragas is an evaluation framework designed to measure the performance of retrieval-augmented generation pipelines and…