Awesome GitHub RepositoriesPerformance Metrics

Autoresearch is an autonomous machine learning research agent and architecture search framework. It employs a closed-loop system to programmatically rewrite training and architecture source code to discover optimal language model configurations. The system iteratively modifies code and evaluates performance metrics to improve model quality based on a target objective. It optimizes model performance and training efficiency by tracking validation bits per byte, which allows for a fair comparison of architectural changes independently of vocabulary size. The framework manages the full training

This project is a comprehensive Chinese translation of a technical deep learning textbook, providing an educational resource on the theory and implementation of neural networks. It functions as a collaborative technical translation project designed to make complex academic AI literature accessible to non-English speakers. The project utilizes a community-driven translation model that integrates external suggestions and pull requests to refine linguistic accuracy and reduce bias. It employs standardized terminology mapping to ensure a uniform vocabulary throughout the translated content. To i

This project is a modular research toolkit designed for developing, training, and evaluating deep learning models for object detection, segmentation, and video instance tracking. It provides a flexible training engine that manages complex neural network execution, including distributed training, custom lifecycle hooks, and weight optimization. The framework is built around a hierarchical configuration system that allows users to define architectures, data pipelines, and training hyperparameters through composable, inheritable files. The project distinguishes itself through its highly modular

This project provides a collection of machine learning algorithms implemented from scratch in Python. It serves as an educational resource using interactive notebooks that combine code with mathematical explanations to demonstrate the first principles of data science. The repository includes reference implementations for neural networks, such as multilayer perceptrons with backpropagation, and supervised learning models including linear and logistic regression. It also covers unsupervised learning through k-means clustering and Gaussian anomaly detection. The codebase covers a broad range of

Audiocraft is a deep learning audio library and machine learning framework designed for training, fine-tuning, and evaluating generative models for music and sound effects. It functions as a text-to-music generative model and a neural audio codec, providing the tools necessary to compress audio signals into discrete representations and synthesize high-fidelity waveforms from textual descriptions. The framework is distinguished by its ability to combine multiple conditioning signals, allowing for the generation of audio based on text prompts, melodic excerpts, or style-based audio clips. It al

Deepeval is a framework for testing and evaluating large language model applications. It provides a suite of tools for executing automated regression tests, validating model output quality against defined standards, and tracing the execution of complex agent workflows. By integrating these capabilities into development pipelines, the platform ensures consistent performance and reliability throughout the software lifecycle. The platform distinguishes itself through its focus on programmatic validation and observability. It utilizes secondary language models to score output quality and employs

Ragas is an evaluation framework designed to measure the performance of retrieval-augmented generation pipelines and autonomous agent workflows. It provides a comprehensive suite of tools for benchmarking system outputs, utilizing language models as automated judges to score performance against defined rubrics and reference data. By standardizing inputs, retrieved contexts, and generated responses into a unified schema, the project enables consistent analysis across complex AI applications. The framework distinguishes itself through its ability to generate synthetic test datasets from existin

FlagEmbedding is a comprehensive toolkit designed for training, benchmarking, and deploying embedding models, retrieval systems, and augmented generation pipelines. It provides the necessary infrastructure to transform text into high-dimensional vector representations and organize them into searchable structures for semantic search applications. The framework distinguishes itself through specialized capabilities for fine-tuning pre-trained embedding and reranking models on domain-specific datasets. By allowing users to adapt models to unique vocabularies and specialized retrieval tasks, it en

TPOT is a Python automated machine learning tool and pipeline framework. It automatically searches, selects, and tunes machine learning algorithms and hyperparameters to identify the most effective model architecture. The system utilizes genetic programming to optimize these pipelines through evolutionary algorithms. To accelerate the search process, it functions as a multi-core evaluator that runs parallel training workflows across multiple processor cores. The framework supports the definition of custom objective functions to optimize pipelines based on specific performance metrics.

PyOD is a Python anomaly detection library used to identify outliers in tabular, time series, graph, text, and image data. It provides a collection of algorithms for detecting anomalous data points and includes a unified detector interface that standardizes input and output signatures across its available detection algorithms. The project features a multi-modal outlier detector for identifying anomalies across diverse formats including unstructured text and images, as well as a specialized toolkit for graph-based and time-series anomaly detection. It includes an ensemble framework for combini

This project is a collection of foundational machine learning algorithms and data science tools implemented in Python. It focuses on building the logic of these tools using basic programming primitives rather than relying on specialized libraries. The implementation covers several core domains, including a linear algebra library for matrix and vector operations, a statistical analysis toolkit for probability and hypothesis testing, and a framework for map-reduce distributed processing. It also includes implementations for natural language processing, graph theory for network analysis, and var

CatBoost is a gradient boosting machine learning library used to train decision tree ensembles for regression, classification, and ranking tasks. It functions as a high-performance framework that provides a categorical data processor for transforming non-numeric features, a distributed trainer for large-scale datasets, and GPU acceleration to speed up model construction. The library distinguishes itself through native handling of categorical data and text features, removing the need for manual encoding. It includes a specialized model interpretability tool that leverages SHAP values and featu

This project is a comprehensive machine learning interview guide and technical study resource designed for individuals preparing for machine learning and AI engineering roles. It provides a collection of materials and practice problems covering core algorithms, theoretical fundamentals, and the implementation of neural network architectures. The resource serves as a technical reference for generative AI development, focusing on the design and optimization of large language models and diffusion systems. It includes frameworks for system design, covering the architecture of production machine l

This project is a comprehensive collection of educational examples and reference implementations for building vision and language models using PyTorch. It serves as a deep learning tutorial covering the end-to-end process of developing neural networks, from initial architecture definition to final production deployment. The repository provides detailed guides on implementing a wide range of domain-specific models, including convolutional neural networks for object detection and segmentation, as well as transformer and recurrent architectures for natural language processing. It emphasizes gene

Evidently is an AI observability platform and evaluation framework designed to quantify the performance of machine learning models and large language models. It functions as a monitoring tool for detecting data drift and quality degradation in tabular datasets, while providing a specialized analyzer for the faithfulness and correctness of retrieval augmented generation systems. The project distinguishes itself through an evaluation framework that utilizes judge models and custom rubrics to score language model outputs. It includes tools for iterative prompt optimization and the generation of

Synaptic is a JavaScript neural network library used for building, training, and executing neural networks in Node.js and the browser. It provides a framework for constructing architecture-free neural network topologies, a backpropagation training engine for weight optimization, and a toolkit for implementing recurrent neural network frameworks. The library enables the design of custom first or second order network architectures without predefined constraints. It supports a variety of specialized models, including Long Short-Term Memory networks, Hopfield networks, Liquid State Machines, and

Anomalib is a PyTorch-based library for visual anomaly detection, offering a modular framework, a comprehensive model zoo, and a benchmarking suite designed for industrial defect detection. It provides a wide range of algorithms—including generative, discriminative, teacher-student, and vision-language approaches—that support unsupervised, few-shot, and zero-shot settings. The library enables deployment through model export to ONNX and OpenVINO for edge devices, and includes a no-code web application for training and inference. It also features a command-line interface for orchestrating multi

Ce projet fournit une version traduite des guides et références API de la bibliothèque de machine learning scikit-learn pour les sinophones. Il sert de base de connaissances localisée et de référence technique pour implémenter l'analyse prédictive de données et la modélisation statistique en utilisant une boîte à outils basée sur Python. La ressource couvre l'implémentation de l'apprentissage supervisé, incluant les tâches de classification et de régression, ainsi que les workflows d'apprentissage non supervisé pour la découverte de motifs et la détection d'anomalies. Elle fournit également des conseils sur l'éducation en data science, en se concentrant spécifiquement sur l'utilisation de scikit-learn pour le machine learning. La documentation inclut des instructions détaillées sur le prétraitement des données, la réduction de dimensionnalité et la sélection de caractéristiques. Elle détaille en outre l'évaluation et le réglage des modèles via des métriques de performance, l'optimisation des hyperparamètres et la validation de généralisation, ainsi que l'utilisation de pipelines de prédiction et d'utilitaires de traitement du langage naturel.

mmocr est un framework de reconnaissance optique de caractères (OCR) basé sur PyTorch conçu pour entraîner et déployer des modèles de détection de texte, de reconnaissance et d'extraction d'informations clés. Il sert de boîte à outils complète pour la détection et la reconnaissance de texte dans les scènes, fournissant des bibliothèques spécialisées pour localiser les régions de texte et convertir le texte visuel en chaînes encodées par machine. Le projet se distingue par un framework de recherche pour l'extraction d'informations clés et des capacités avancées de repérage de texte. Celles-ci incluent le repérage basé sur des points utilisant des transformers et l'utilisation de courbes de Bezier paramétrées pour identifier et transcrire du texte de forme arbitraire. Le framework couvre une large surface de capacités de vision par ordinateur, notamment la gestion de pipeline de données pour augmenter et standardiser divers jeux de données OCR, l'entraînement de modèles avec mise à l'échelle distribuée et l'évaluation des performances utilisant des métriques OCR standard. Il fournit également des utilitaires pour la manipulation de polygones géométriques et la visualisation des résultats pour auditer les prédictions par rapport aux annotations de vérité terrain. Le système est implémenté en Python et prend en charge l'installation via l'empaquetage d'environnement Docker.