6 रिपॉजिटरी
Awesome GitHub RepositoriesBenchmark Data Generators
Functions for creating deterministic datasets based on industry benchmarks.
Distinct from Benchmarks: Distinct from Benchmarks: focuses on the generation of data rather than the metrics themselves.
Explore 6 awesome GitHub repositories matching testing & quality assurance · Benchmark Data Generators. Refine with filters or upvote what's useful.
Awesome Benchmark Data Generators GitHub Repositories
AI के साथ बेहतरीन रिपॉजिटरी खोजें।हम AI का उपयोग करके सबसे सटीक रिपॉजिटरी खोजेंगे।
- prestodb/presto
prestodb/presto
16,711GitHub पर देखेंPresto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing
Generates deterministic benchmark datasets for performance testing.
Javabig-datadatahadoop - gunnarmorling/1brc
gunnarmorling/1brc
8,062GitHub पर देखेंThe 1BRC (One Billion Row Challenge) is a Java performance benchmarking exercise that processes one billion temperature records from a text file to compute the minimum, mean, and maximum temperature per weather station. At its core, it is a large-scale data aggregation challenge designed to test how efficiently a Java program can parse and aggregate structured data from a plain text file, serving as both a programming exercise and a benchmark for Java performance optimization. The project distinguishes itself through a collection of performance-oriented architectural patterns for high-through
Creates a file with a configurable number of random temperature measurements for benchmarking.
Java1brcchallenges - vespa-engine/vespa
vespa-engine/vespa
6,961GitHub पर देखेंVespa is a distributed search engine, vector database, and machine learning ranking engine. It serves as an AI search platform designed to handle large-scale document indexing and complex query processing across a cluster of nodes, combining keyword retrieval with high-dimensional embedding storage for semantic similarity search. The platform distinguishes itself by integrating machine learning models directly into the search pipeline to perform real-time inference and ranking. It converts these models into ranking expressions to score and order results based on relevance, while providing a s
Includes tools to extract query URLs from logs and generate deterministic datasets for performance benchmarking.
Java - bheisler/criterion.rs
bheisler/criterion.rs
5,485GitHub पर देखेंCriterion रस्ट के लिए एक सांख्यिकी-संचालित माइक्रोबेंचमार्किंग लाइब्रेरी और परफॉरमेंस रिग्रेशन टूल है। यह छोटे कोड सेगमेंट को अलग करने और मापने के लिए एक फ्रेमवर्क प्रदान करता है, जो शोर को खत्म करने और निष्पादन गति के विश्वसनीय, दोहराने योग्य माप सुनिश्चित करने के लिए सांख्यिकीय विश्लेषण का उपयोग करता है। यह टूल एक परफॉरमेंस विज़ुअलाइज़ेशन सूट के माध्यम से खुद को अलग करता है जो परफॉरमेंस ट्रेंड्स और थ्रूपुट को ट्रैक करने के लिए HTML रिपोर्ट और ग्राफ़ उत्पन्न करता है। इसमें परफॉरमेंस में गिरावट की पहचान करने और उसे रोकने के लिए वर्तमान निष्पादन समय की तुलना संग्रहीत बेसलाइन के साथ करने की एक प्रणाली शामिल है। यह लाइब्रेरी एसिंक्रोनस फ़ंक्शन मापन, इनपुट स्केलिंग के लिए पैरामीटराइज़्ड बेंचमार्किंग और कोड थ्रूपुट गणना को कवर करती है। यह रन के दौरान निम्न-स्तरीय डेटा कैप्चर करने के लिए कस्टम हार्डवेयर मेट्रिक्स और प्रोसेसर काउंटरों के एकीकरण का भी समर्थन करती है। ऑटोमेशन को बेंचमार्क फ़िल्टरिंग के लिए कमांड-लाइन इंटरफ़ेस और निरंतर एकीकरण (CI) पाइपलाइनों के भीतर सफल निष्पादन को सत्यापित करने के लिए एक वैलिडेशन मोड के माध्यम से समर्थित किया जाता है।
Transforms benchmark data into compatible formats suitable for statistical analysis and summary reporting.
Rustbenchmarkcriteriongnuplot - swe-bench/swe-bench
SWE-bench/SWE-bench
4,321GitHub पर देखेंSWE-bench is an automated evaluation framework that tests large language models on real-world software engineering tasks. It measures how effectively models can generate and apply code patches that resolve actual GitHub issues, using a standardized dataset and scoring system built around Docker-based patch verification against original project test suites. The framework provides curated benchmark datasets spanning comprehensive, fast, verified, multilingual, and multimodal evaluation splits, allowing targeted assessment of model capabilities across different programming languages and issue ty
Runs a data collection procedure on user-provided repositories to generate new evaluation instances.
Pythonbenchmarklanguage-modelsoftware-engineering - transformerlensorg/transformerlens
TransformerLensOrg/TransformerLens
3,098GitHub पर देखेंTransformerLens is a library for mechanistic interpretability research designed to reverse engineer the learned algorithms within large language models. It provides a standardized framework for wrapping diverse transformer architectures, allowing researchers to extract, manipulate, and analyze internal activations and weights through a consistent interface. The project distinguishes itself through a comprehensive system of activation hooks that can capture, patch, and ablate internal tensors during the forward pass. It includes specialized utilities for decomposing fused projections, material
Generates specialized datasets for tasks like Indirect Object Identification to benchmark learned algorithms.
Python
