Awesome GitHub RepositoriesVector and AI Data Pipelines

Pathway is a high-performance data processing framework designed for building unified batch and streaming pipelines. It functions as an orchestrator for complex data transformations, utilizing a differential dataflow engine to process updates incrementally. By treating static datasets and continuous event streams with identical logic, the platform ensures exactly-once processing semantics and consistent results across diverse data sources. The framework distinguishes itself through its specialized support for real-time artificial intelligence and retrieval-augmented generation. It features in

Ultralytics is a comprehensive computer vision framework designed for training, validating, and deploying deep learning models across a wide range of visual recognition tasks. It provides a unified interface for core operations including object detection, instance segmentation, pose estimation, and image classification. By utilizing a modular architecture, the platform allows users to swap model components to balance inference speed and accuracy requirements for diverse applications. The framework distinguishes itself through its support for real-time processing and flexible deployment. It in

labelImg este un instrument desktop de adnotare a imaginilor și un utilitar de pregătire a seturilor de date utilizat pentru a crea seturi de date etichetate pentru antrenarea viziunii computerizate. Acesta oferă o interfață grafică pentru desenarea casetelor de delimitare în jurul obiectelor din imagini și atribuirea lor de etichete de clasă pentru a construi date de referință (ground truth) pentru modelele de învățare automată. Software-ul suportă în mod specific formatul de adnotare Pascal VOC XML, exportând coordonatele imaginilor și numele claselor în structuri XML sau text standard. Acesta permite utilizatorilor să încarce liste de clase predefinite din fișiere text pentru a standardiza denumirea în cadrul întregului proiect. Dincolo de etichetarea inițială, instrumentul acoperă fluxuri de lucru de adnotare a imaginilor, inclusiv vizualizarea adnotărilor salvate și verificarea manuală a setului de date. Aceasta include capacitatea de a marca imaginile ca fiind verificate sau dificile pentru a menține calitatea setului de date.

Bullet3 is a professional physics simulation engine designed for calculating rigid body, soft body, and collision dynamics within 3D environments and robotics applications. It functions as a computational framework for determining complex geometric intersections and contact manifolds between objects in simulated space. The library distinguishes itself through a distributed rendering framework that scales heavy graphical workloads and scene generation tasks across large clusters of machines. This capability enables the production of massive datasets by distributing complex scene generation acr

DataHub is a metadata management platform designed to unify technical, operational, and business context across diverse data ecosystems. By utilizing a graph-based metadata model and an event-driven ingestion architecture, it creates a centralized source of truth that maps complex data relationships, lineage, and ownership. This foundational framework enables organizations to maintain a synchronized view of their data landscape, supporting both human-led discovery and automated data operations. The platform distinguishes itself through its focus on grounding artificial intelligence and autono

This project is a collection of educational resources and implementation frameworks providing deep learning model recipes, code samples, and step-by-step guides for computer vision tasks. It organizes complex workflows into modular recipes and implementation guides to facilitate the building of image and video analysis models. The framework focuses on specialized vision capabilities, including an image similarity framework for fast retrieval and re-ranking, human pose estimation, and video action recognition. It also provides specific tools for crowd density estimation and document image clea

Reader is an AI data ingestion pipeline and web content parser designed to convert websites and documents into clean markdown for use with large language models. It functions as a headless browser content extractor and web-to-markdown converter, transforming URLs and PDF files into structured text formats while removing irrelevant web clutter. The system optimizes retrieval augmented generation by acting as a search optimizer that retrieves web results and applies re-ranking to improve context relevance. It further enhances content accessibility by using vision models to generate descriptive

ClearML is a comprehensive MLOps platform designed to manage the entire machine learning lifecycle. It functions as an experiment tracking tool, a data versioning system, and a pipeline orchestrator, while providing infrastructure for GPU cluster management and model serving. The platform is distinguished by its ability to handle hybrid-cloud compute scheduling and fractional GPU allocation, allowing multiple workloads to share a single hardware accelerator. It employs a metadata-based approach to data versioning, using virtual views to track large datasets and artifacts without duplicating r

RavenDB is a multi-model NoSQL document database designed for high-performance, ACID-compliant data storage. It persists structured information as schema-flexible JSON documents and utilizes a unit-of-work session pattern to track entity changes and batch modifications into atomic transactions. The platform is built on a distributed architecture that supports horizontal scaling through sharding and ensures high availability via multi-node, master-to-master cluster replication. The database distinguishes itself through a self-optimizing query engine that automatically creates and maintains ind

models.dev is a directory and intelligence system for large language models that provides a standardized catalog of technical specifications, provider mappings, and pricing data. It serves as a central index for model metadata, including context windows, output limits, and release dates. The project functions as a capability index and pricing comparison tool, allowing for the analysis of token costs across different hosting providers. It maps generic model names to the specific API identifiers required by various third-party platforms and tracks support for functional features such as tool ca

This project is a deep learning system designed for real-time emotion recognition and facial expression analysis. It utilizes a convolutional neural network architecture to process raw visual input, mapping complex facial patterns to seven distinct emotional states through a supervised machine learning pipeline. The system functions as both a training framework and an inference engine. It includes utilities for preparing and standardizing large image datasets to ensure consistent input quality, alongside a real-time processing pipeline that captures and buffers live video frames to perform co