cocoindex-io/cocoindex

Features

• Code Context Servers - Serves a live index of the entire codebase to coding agents through MCP, CLI, or Claude skills.
• Data Lineage - Tags every output record with source items and transformation version for full provenance tracking.
• Knowledge Graph Extraction - Parses unstructured text to identify entities, relationships, and statements, storing them as a queryable knowledge graph.
• Incremental Processing - Ships an incremental processing engine that re-computes only changed data for sub-second freshness.
• Delta Processing - Re-processes only changed source records or code paths, avoiding full recomputation for sub-second freshness.
• Python-Defined Transformations - Defines data transformations as pure Python functions and automatically derives the execution graph.
• Target State Declarations - Implements declarative target-state reconciliation where users describe the desired end state and the engine computes minimal mutations.
• Declarative Data Reconciliation Pipelines - Provides declarative pipelines that reconcile data stores to a desired state with minimal mutations.
• Incremental Computation - Processes data changes incrementally so only modified content is re-computed, keeping large corpora fresh without full recomputation.
• Codebase Indexing - Provides incremental codebase indexing with file-granularity change tracking and AST-aware parsing for AI agents.
• Knowledge Graph Construction Tools - Builds structured knowledge graphs from documents for AI agent reasoning.
• Real-Time - Builds real-time knowledge graphs with incremental updates and high-performance graph queries from documents.
• Multi-Modal RAG Pipelines - Provides a complete pipeline for building multi-modal RAG systems with lineage tracking.
• Builders - Provides a declarative engine for building semantic text indexes with embeddings and natural language querying.
• Continuous Sync Engines - Provides a continuous sync engine that watches source changes and keeps derived indexes fresh.
• Code Search - Provides natural-language code search with syntax-aware chunking for AI agent context retrieval.
• Schema Inference - Infers output schemas automatically from transformation pipeline type annotations and function signatures.
• Incremental Vector Sync - Keeps vector indexes continuously updated by processing only the delta from live sources.
• Codebase Indexing - Maintains a live, shared index of source code that updates automatically with each commit for team agents.
• AI Agent Indexes - Provides incremental codebase indexing with AST-aware parsing for AI agent context.
• AST-Aware Parsers - Provides AST-aware parsing to index codebases for AI agent context.
• Shared Code Index Daemons - Runs a persistent daemon that builds a single code index and serves it to every teammate and agent.
• Delta-Only Pipeline Executors - Runs only the delta of changed data and code, skipping unchanged work at any scale.
• Granular File Change Tracking - Maps each source file to its own processing component for independent transformation and precise delta detection.
• Incremental Data Pipelines - Ships an incremental data pipeline engine that processes only changed data for sub-second freshness at any scale.
• Data Store Reconciliation - Computes minimal mutations to reconcile a declared target data store state with the current state.
• Execution Graphs - Composes user-defined Python functions into a directed acyclic graph for scheduled execution with automatic parallelism.
• Data Provenance Recorders - Records the provenance of every output byte, tracing results back to source records and transformation steps.
• Automatic ML Workload Batching - Automatically batches GPU and ML workloads like text embeddings for higher throughput.
• Multi-Vector Embeddings - Natively handles typed multi-dimensional vectors, from simple arrays to multi-vector embeddings for multimodal AI pipelines.
• Event-Driven Transform Pipelines - Builds real-time transformation pipelines triggered by events from object storage and message queues.
• Incremental File Converters - Converts source files to target formats incrementally, reprocessing only changed or new files.
• Data Consistency Models - Implements data consistency models to ensure indexes converge correctly during concurrent updates.
• Data Ingestion Sources - Reads data from any external system and keeps it incrementally fresh as knowledge for AI agents.
• Data Destination Connectors - Exports indexed data to any destination including local files, cloud storage, or REST APIs.
• Multi-Vector Embedding Handlers - Natively handles typed multi-dimensional vectors from simple arrays to multi-vector embeddings for multimodal AI pipelines.
• File-Level Pipeline Definitions - Maps each source file to its own processing component for independent transformation and precise delta detection.
• Self-Updating Documentation Generators - Generates and incrementally updates wiki pages for each project in a codebase.
• Index Scaling - Distributes indexing work across nodes with elastic, fault-tolerant scaling for petabyte-scale corpora.
• Elastic - Distributes indexing work across nodes with fault-tolerant scaling for petabyte-scale corpora.
• Unified Structured-Unstructured Indexes - Builds a unified, incrementally updated search index over both structured and unstructured data.
• Enterprise Source Indexing - Ingests and indexes content from codebases, meetings, Slack, docs, and tickets for live AI agent context.
• Branch Delta Indexing - Re-indexes only files that differ between branches, layering changes on a shared main index to save compute.
• Concurrency Controllers - Controls concurrency of data-processing operations to optimize performance and prevent system overload.
• Cross-Repository Indexes - Provides a daemon that indexes multiple repositories for cross-repo code queries.
• Data Pipeline Lineage Inspectors - Provides a tool to inspect, trace, and debug every step of a data pipeline in real time.
• Search Result Provenance Tracers - Traces search results back to source data to debug and improve the indexing strategy.
• Large File Processing - Processes large files during data indexing by managing granularity, fan-in, fan-out, and memory pressure.
• RAG Frameworks - ETL framework for indexing data with real-time incremental updates.
• Data Pipelines - High-performance data transformation framework for AI workflows.
• Stream Processing - ETL framework designed for building fresh indices for AI applications.
• Streaming Engines - ETL framework for building real-time AI indexes.
• Workflow Frameworks - ETL framework for building fresh data indexes.
• General Productivity Tools - AI-powered search and indexing for codebases.

Star history