Marker-Inc-Korea/AutoRAG

Features

• Retrieval Augmented Generation Pipelines - Provides a modular framework for constructing and optimizing end-to-end retrieval-augmented generation pipelines.
• RAG Pipeline Optimizers - Provides tools for automating the search and selection of the most effective retrieval and generation modules for RAG pipelines.
• RAG Evaluation Dataset Generation - Automatically generates question-answer pairs and ground truth from knowledge sources to benchmark RAG performance.
• RAG Evaluation Dataset Generators - Generates synthetic question-answer pairs from raw documents to establish ground truth benchmarks for evaluation.
• LLM Test Pair Generators - Generates synthetic question-answer pairs from document corpora using language models to establish evaluation benchmarks.
• Document Chunking Strategies - Divides parsed documents into segments using semantic or fixed-length strategies to optimize retrieval performance.
• RAG Pipelines - Allows connecting retrieval modules, prompt builders, and generators in a sequential chain.
• Document Chunking & Embedding - Splits extracted text into optimized segments and manages document preparation for retrieval-augmented generation.
• Automated ML (AutoML) - Implements automated selection and hyperparameter tuning of RAG modules to identify the most effective architectural configuration.
• RAG Architecture Optimization - Implements an automation layer that applies machine learning search strategies to optimize RAG architectures.
• RAG Evaluation Frameworks - Provides a framework for measuring RAG pipeline performance using specialized retrieval and generation metrics.
• Synthetic Dataset Generation - Generates synthetic question-answer pairs and ground truth corpora from raw documents for pipeline benchmarking.
• Document Parsing and Extraction - Converts various file formats like PDF and HTML into structured text suitable for LLM ingestion.
• Document Parsing Pipelines - Implements automated routines to parse diverse file formats like PDF and XML into structured text chunks.
• Document Parsing Engines - Integrates multiple OCR and text extraction engines to convert multimodal documents into structured formats.
• Structured Text Converters - Converts raw documents into metadata-rich text, Markdown, or JSON formats optimized for LLM analysis.
• Node Performance Evaluators - Provides metric-based evaluation of individual pipeline components to isolate performance bottlenecks.
• RAG Performance Benchmarks - Measures RAG workflow accuracy using specialized retrieval and generation metrics to identify optimal hyperparameters.
• RAG Configuration Experiments - Runs automated experiments across various module combinations to find the best configuration for a specific dataset.
• Grounded Answer Generation - Creates reference answers for questions based on provided context using a language model to establish baselines.
• Evaluation Visualizers - Displays trial outcomes and performance metrics through a dashboard to analyze the optimization process.
• Grid Search Executors - Performs exhaustive grid searches over predefined module versions and settings to optimize the retrieval-augmented generation pipeline.
• Multimodal Document Parsers - Uses vision-capable models to parse complex documents by analyzing visual elements alongside text.
• Custom Sentence Splitting - Integrates external tokenizers and custom splitting logic to support specialized text segmentation and multiple languages.
• RAG Performance Metrics - Sets specific performance thresholds and metrics to determine the effectiveness of individual pipeline nodes.
• OCR Document Parsers - Utilizes optical character recognition services to extract text from raw document images page by page.
• Hybrid Table Detectors - Combines text and table parsing modules to detect tabular data on a per-page basis.
• RAG Production Deployments - Allows exporting verified pipeline configurations into API servers or scripts for real-world production use.
• Configuration-Driven Pipelines - Uses YAML configuration files to define pipeline structures and parameters for reproducible evaluation and deployment.
• Pipeline Performance Evaluators - Runs automated trials against datasets to measure retrieval and generation metrics for configuration identification.
• Application Development - AutoML tool for optimizing RAG answer quality.
• Automated Machine Learning - AutoML tool specifically for optimizing RAG pipeline performance.
• Knowledge Retrieval - AutoML tool for finding optimal RAG pipelines.
• Retrieval Augmented Generation - AutoML tool for finding optimal retrieval pipeline configurations.

Star history