Awesome GitHub RepositoriesDataset Management

Ultralytics is a comprehensive computer vision framework designed for training, validating, and deploying deep learning models across a wide range of visual recognition tasks. It provides a unified interface for core operations including object detection, instance segmentation, pose estimation, and image classification. By utilizing a modular architecture, the platform allows users to swap model components to balance inference speed and accuracy requirements for diverse applications. The framework distinguishes itself through its support for real-time processing and flexible deployment. It in

YOLOv5 is a comprehensive computer vision framework designed for end-to-end deep learning, specializing in real-time object detection, image classification, and instance segmentation. It provides a unified toolkit that manages the entire lifecycle of a model, from initial dataset configuration and hyperparameter tuning to high-speed inference and deployment. The framework utilizes a modular neural architecture, allowing users to swap backbone and head components to tailor models for specific visual tasks. What distinguishes this project is its focus on production-ready deployment and model ef

Supervision is a computer vision toolset for normalizing model outputs, managing datasets, and visualizing annotations. It provides a framework to convert predictions from various classification and detection models into a standardized data format to ensure interoperability across different computer vision pipelines. The library features a post-processor for filtering, counting, and tracking detected objects across image frames and video streams. It includes capabilities for large image tiling to improve the detection of small objects and tools for assigning persistent identities to objects t

This project is a comprehensive research platform designed for the end-to-end lifecycle of robotic learning. It provides a modular framework for training neural network policies—specifically through imitation and reinforcement learning—and deploying them onto physical robotic hardware. By offering a unified interface for hardware abstraction, the platform decouples high-level control logic from the specific sensors and actuators of diverse robotic systems. The framework distinguishes itself through a standardized approach to data and policy management. It utilizes a consistent schema for reco

CVAT ist ein Open-Source-Annotationstool für Computer Vision und eine Plattform zur Verwaltung visueller Datensätze. Es bietet eine selbst gehostete Schnittstelle zum Labeln von Bildern, Videos und 3D-Daten, um Datensätze für Vision-KI-Modelle zu erstellen. Die Plattform bietet KI-gestützte Daten-Labeling-Funktionen zur Automatisierung der Erstellung von Masken und Bounding Boxes und nutzt ein Plug-in-System zur Anbindung externer Modelle für maschinelles Lernen. Sie enthält ein konsensbasiertes Qualitätssicherungssystem, das die Genauigkeit von Labels durch den Vergleich unabhängiger Annotationen überprüft. Das System deckt kollaboratives Teammanagement, Projektorganisation durch Aufgabenzerlegung und die Integration von Remote-Cloud-Speichern ab. Es bietet zudem eine REST-API für die programmatische Workflow-Steuerung sowie den Import und Export von Daten in branchenüblichen Formaten.

CVAT is an open-source, web-based platform designed for annotating images, videos, and 3D point clouds to create high-quality training datasets for machine learning. It functions as a containerized server that orchestrates the entire lifecycle of computer vision data, from initial task creation and manual labeling to quality assurance and final dataset export. The platform distinguishes itself through deep integration with machine learning models, allowing users to deploy custom AI models as serverless functions for automated object detection, tracking, and skeleton annotation. It supports co

This project is a comprehensive toolkit designed for the full lifecycle management of large language and multimodal models. It functions as a unified orchestrator that handles the entire development process, ranging from dataset preparation and supervised fine-tuning to advanced reinforcement learning alignment and production-ready inference deployment. The platform distinguishes itself through a specialized reinforcement learning library that supports complex optimization algorithms, including group relative policy optimization and leave-one-out techniques, to improve model instruction-follo

Easy-dataset is a comprehensive platform designed for the end-to-end management of machine learning datasets, specifically tailored for language and vision model fine-tuning. It functions as a centralized environment for the entire data lifecycle, encompassing the automated generation of synthetic training data, the structural organization of document collections, and the systematic annotation of individual data points. The platform distinguishes itself through its integrated evaluation and orchestration capabilities. It provides a dedicated suite for benchmarking models, featuring blind side

Axolotl is a configuration-driven framework designed for the fine-tuning, evaluation, and quantization of large language models. It functions as a comprehensive orchestrator for distributed training, enabling users to manage complex workflows across multi-node and multi-GPU environments. By utilizing structured configuration files, the platform streamlines the setup of training parameters, dataset paths, and hardware distribution strategies. The project distinguishes itself through its support for diverse training methodologies, including full-parameter tuning, parameter-efficient adaptation,

Doccano is a collaborative data labeling platform and machine learning dataset management system. It provides a web-based interface for teams to import raw text, mark datasets, and export structured annotations for model training. The project specifically supports text annotation for classification and named entity recognition tasks. It enables teams to coordinate multiple users on a single project to maintain consistent labeling guidelines and increase the speed of dataset creation. The system includes tools for data management and team coordination, providing the ability to import raw data

Moto is a cloud service mockery framework and API mock server that simulates AWS infrastructure locally. It allows developers to test cloud-dependent code and verify infrastructure-as-code templates without deploying real resources or incurring costs. The project functions as an SDK interceptor that can patch existing service clients to redirect requests to a local mock environment. It can also be run as a standalone HTTP server, enabling any programming language to interact with the simulated endpoints. The framework covers a vast array of simulated capabilities, including data storage, com

MMPose is a PyTorch-based pose estimation toolbox and deep learning training pipeline designed for detecting 2D and 3D keypoints on humans, animals, and faces. It serves as a computer vision model zoo and a framework for both 2D pose estimation and 3D pose lifting. The project is distinguished by its modular architecture and extensibility, employing a registry-based system and hierarchical configurations to allow for custom algorithm integration and model pipeline customization. It supports diverse estimation paradigms, including top-down, bottom-up, and two-stage pose lifting workflows. The

sd-scripts is a suite of utilities designed for fine-tuning generative models, preprocessing datasets, and converting model weights. It provides a collection of scripts for executing Stable Diffusion training through methods such as DreamBooth, textual inversion, and full fine-tuning, alongside a framework for creating and managing Low-Rank Adaptation weights. The project features specialized capabilities for model weight conversion between different architectures and precision formats. It includes tools for merging adaptation weights into base models, extracting weights from trained models,

mmocr ist ein auf PyTorch basierendes Framework für optische Zeichenerkennung (OCR), das für das Training und Deployment von Modellen zur Texterkennung, -identifizierung und Extraktion von Schlüsselinformationen entwickelt wurde. Es dient als umfassende Toolbox für die Erkennung und Identifizierung von Text in Szenen und bietet spezialisierte Bibliotheken zum Lokalisieren von Textregionen und zum Konvertieren von visuellem Text in maschinell kodierte Strings. Das Projekt zeichnet sich durch ein Forschungs-Framework für die Extraktion von Schlüsselinformationen und fortgeschrittene Text-Spotting-Funktionen aus. Dazu gehören punktbasiertes Spotting mittels Transformern und die Verwendung parametrisierter Bezier-Kurven, um beliebig geformten Text zu identifizieren und zu transkribieren. Das Framework deckt ein breites Spektrum an Computer-Vision-Funktionen ab, einschließlich Daten-Pipeline-Management zur Augmentierung und Standardisierung diverser OCR-Datensätze, Modelltraining mit verteilter Skalierung und Performance-Evaluierung unter Verwendung von Standard-OCR-Metriken. Es bietet zudem Dienstprogramme für geometrische Polygon-Manipulation und Ergebnisvisualisierung zur Überprüfung von Vorhersagen gegen Ground-Truth-Annotationen. Das System ist in Python implementiert und unterstützt die Installation über Docker-Umgebungs-Packaging.

This project is a dataset management framework and cross-framework data loader that provides a unified interface for reading data formats compatible with TensorFlow, JAX, and PyTorch. It serves as a library of curated public datasets provided as data streams and includes tools for building, versioning, and documenting large-scale datasets. The system differentiates itself through a distributed data processing engine capable of managing massive datasets across clusters using parallelized pipelines. It utilizes builder-based construction to standardize how data is downloaded and prepared, while