6 repository-uri
Awesome GitHub RepositoriesBenchmark Data Generators
Functions for creating deterministic datasets based on industry benchmarks.
Distinct from Benchmarks: Distinct from Benchmarks: focuses on the generation of data rather than the metrics themselves.
Explore 6 awesome GitHub repositories matching testing & quality assurance · Benchmark Data Generators. Refine with filters or upvote what's useful.
Awesome Benchmark Data Generators GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- prestodb/presto
prestodb/presto
16,711Vezi pe GitHubPresto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing
Generates deterministic benchmark datasets for performance testing.
Javabig-datadatahadoop - gunnarmorling/1brc
gunnarmorling/1brc
8,062Vezi pe GitHubThe 1BRC (One Billion Row Challenge) is a Java performance benchmarking exercise that processes one billion temperature records from a text file to compute the minimum, mean, and maximum temperature per weather station. At its core, it is a large-scale data aggregation challenge designed to test how efficiently a Java program can parse and aggregate structured data from a plain text file, serving as both a programming exercise and a benchmark for Java performance optimization. The project distinguishes itself through a collection of performance-oriented architectural patterns for high-through
Creates a file with a configurable number of random temperature measurements for benchmarking.
Java1brcchallenges - vespa-engine/vespa
vespa-engine/vespa
6,961Vezi pe GitHubVespa is a distributed search engine, vector database, and machine learning ranking engine. It serves as an AI search platform designed to handle large-scale document indexing and complex query processing across a cluster of nodes, combining keyword retrieval with high-dimensional embedding storage for semantic similarity search. The platform distinguishes itself by integrating machine learning models directly into the search pipeline to perform real-time inference and ranking. It converts these models into ranking expressions to score and order results based on relevance, while providing a s
Includes tools to extract query URLs from logs and generate deterministic datasets for performance benchmarking.
Java - bheisler/criterion.rs
bheisler/criterion.rs
5,485Vezi pe GitHubCriterion este o bibliotecă de microbenchmarking bazată pe statistici și un instrument de regresie a performanței pentru Rust. Oferă un framework pentru izolarea și măsurarea segmentelor mici de cod, utilizând analiza statistică pentru a elimina zgomotul și a asigura măsurători fiabile și repetabile ale vitezei de execuție. Instrumentul se distinge printr-o suită de vizualizare a performanței care generează rapoarte HTML și grafice pentru a urmări tendințele de performanță și throughput-ul. Include un sistem pentru compararea timpilor de execuție actuali cu bazele de referință stocate pentru a identifica și preveni scăderile de performanță. Biblioteca acoperă măsurarea funcțiilor asincrone, benchmarking-ul parametrizat pentru scalarea input-ului și calculul throughput-ului codului. De asemenea, suportă integrarea metricilor hardware personalizate și a contoarelor de procesor pentru a captura date de nivel scăzut în timpul rulărilor. Automatizarea este susținută printr-o interfață CLI pentru filtrarea benchmark-urilor și un mod de validare pentru verificarea execuției reușite în pipeline-urile CI.
Transforms benchmark data into compatible formats suitable for statistical analysis and summary reporting.
Rustbenchmarkcriteriongnuplot - swe-bench/swe-bench
SWE-bench/SWE-bench
4,321Vezi pe GitHubSWE-bench is an automated evaluation framework that tests large language models on real-world software engineering tasks. It measures how effectively models can generate and apply code patches that resolve actual GitHub issues, using a standardized dataset and scoring system built around Docker-based patch verification against original project test suites. The framework provides curated benchmark datasets spanning comprehensive, fast, verified, multilingual, and multimodal evaluation splits, allowing targeted assessment of model capabilities across different programming languages and issue ty
Runs a data collection procedure on user-provided repositories to generate new evaluation instances.
Pythonbenchmarklanguage-modelsoftware-engineering - transformerlensorg/transformerlens
TransformerLensOrg/TransformerLens
3,098Vezi pe GitHubTransformerLens is a library for mechanistic interpretability research designed to reverse engineer the learned algorithms within large language models. It provides a standardized framework for wrapping diverse transformer architectures, allowing researchers to extract, manipulate, and analyze internal activations and weights through a consistent interface. The project distinguishes itself through a comprehensive system of activation hooks that can capture, patch, and ablate internal tensors during the forward pass. It includes specialized utilities for decomposing fused projections, material
Generates specialized datasets for tasks like Indirect Object Identification to benchmark learned algorithms.
Python
