8 dépôts
Automated suites for measuring machine learning model speed and accuracy.
Distinguishing note: Focuses on ML-specific performance metrics rather than general software testing.
Explore 8 awesome GitHub repositories matching testing & quality assurance · Model Benchmarks. Refine with filters or upvote what's useful.
Aider is a command-line interface tool that enables large language models to directly edit, refactor, and manage source code within a local repository. It functions as an AI-powered coding assistant that integrates into the developer workflow, allowing users to apply code changes through natural language prompts while maintaining repository context and version control. The tool distinguishes itself through a specialized diff-based patching engine that parses model-generated search-and-replace blocks to modify specific file segments without rewriting entire files. It features a provider-agnost
Evaluates models on their ability to follow instructions and perform code edits using standardized coding exercises.
This project is a modular research toolkit designed for developing, training, and evaluating deep learning models for object detection, segmentation, and video instance tracking. It provides a flexible training engine that manages complex neural network execution, including distributed training, custom lifecycle hooks, and weight optimization. The framework is built around a hierarchical configuration system that allows users to define architectures, data pipelines, and training hyperparameters through composable, inheritable files. The project distinguishes itself through its highly modular
The project enables evaluating object detection and instance segmentation model robustness by testing performance against various image corruptions and severity levels using analysis scripts.
InsightFace is a comprehensive deep learning framework designed for face recognition, biometric identity verification, and feature extraction. It provides a specialized engine for one-to-one verification and one-to-many identification tasks, utilizing convolutional neural networks to transform raw image pixels into high-dimensional vector embeddings. The project includes a complete toolkit for detecting, aligning, and processing facial data to ensure consistent identity discrimination. Beyond core recognition, the platform distinguishes itself through an extensive model management and optimiz
Comparing the accuracy and speed of different machine learning models to select the best solution for specific hardware constraints.
Modular is a unified machine learning development platform designed for building, compiling, and deploying high-performance neural network models. It provides a comprehensive execution engine that supports both local and production-grade inference, enabling developers to manage the entire model lifecycle from initial architecture definition to scalable, containerized service deployment. The platform distinguishes itself through a hardware-agnostic runtime that abstracts diverse silicon architectures, allowing models to execute efficiently across varied compute environments. It includes a spec
Provides automated tools to measure model speed and accuracy against defined datasets.
Obliteratus is a weight ablation framework and refusal removal tool designed to identify and delete the internal representations responsible for content refusals in large language models without retraining. It functions as a circuit analysis suite that maps the geometric structure of model guardrails to isolate the specific layers and attention heads that enforce refusals. The project enables the removal of these behaviors through geometric projection, rank-1 adapter ablation for reversible modifications, and the application of steering vectors to alter behavior during inference. It includes
Measures perplexity and refusal rates to ensure the model retains general capabilities after internal modification.
This is a PyTorch-based training pipeline designed for reproducible image classification benchmarking on the CIFAR-10 dataset. It integrates GPU-accelerated computation, data augmentation, learning rate scheduling, and checkpointing to produce consistent accuracy measurements across multiple ResNet architectures. The project distinguishes itself by providing a fixed-architecture benchmark suite that trains a predefined set of ResNet variants, from ResNet18 through ResNet152, on CIFAR-10. It implements a step-based learning rate decay schedule at predetermined epochs to stabilize convergence,
Reports the top-1 classification accuracy of multiple pre-configured architectures on the CIFAR-10 test set.
Automatically tests model configurations to find the optimal performance and accuracy for a specific edge device.
MobileModels est une base de données de matériel mobile et un registre d'identification de modèles. Il sert de collection structurée qui mappe les noms commerciaux grand public aux numéros de modèle des fabricants et aux noms de code matériels internes. Le projet fonctionne comme un répertoire de variantes matérielles, distinguant les différentes versions régionales et opérateurs du même appareil parmi diverses marques. Il inclut un tracker de bootloader et de noyau pour surveiller la disponibilité des versions officielles du code source du noyau et le statut de déverrouillage du bootloader. Le registre couvre le catalogage du matériel mobile via le mappage des identifiants d'appareils et l'analyse des conventions de nommage des fabricants pour décoder les numéros de modèle. Il fournit une référence pour lier les spécifications matérielles et les SKU régionaux à leurs noms de code de développement correspondants.
Displays detailed hardware component information, including model numbers and codenames, for system identification.