awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
gunnarmorling avatar

gunnarmorling/1brc

0
View on GitHub↗
8,062 stars·2,234 forks·Java·Apache-2.0·14 viewswww.morling.dev/blog/one-billion-row-challenge↗

1brc

The 1BRC (One Billion Row Challenge) is a Java performance benchmarking exercise that processes one billion temperature records from a text file to compute the minimum, mean, and maximum temperature per weather station. At its core, it is a large-scale data aggregation challenge designed to test how efficiently a Java program can parse and aggregate structured data from a plain text file, serving as both a programming exercise and a benchmark for Java performance optimization.

The project distinguishes itself through a collection of performance-oriented architectural patterns for high-throughput data processing. These include branchless temperature parsing using bitwise operations, CPU-core-local aggregation maps that eliminate lock contention, a custom primitive hash map with long keys and int values to minimize object overhead, and garbage-collection-aware allocation that pre-allocates all working data structures upfront. Additional differentiators include JIT-friendly loop unrolling, memory-mapped file I/O, parallel stream processing across file chunks, and direct memory access via sun.misc.Unsafe to bypass bounds checks.

The project also provides supporting capabilities for benchmarking and profiling, including synthetic dataset generation with configurable parameters for reproducible testing, CPU profiling with flamegraphs to visualize execution time distribution, and tools for measuring and optimizing Java code execution speed against the fixed data processing challenge. The repository includes utilities for generating benchmark data files and profiling application performance to identify bottlenecks.

Features

  • Data Aggregation Challenges - A programming exercise that processes one billion temperature records from a text file to compute per-station statistics.
  • Performance Benchmarks - Measuring and optimizing the execution speed of Java programs processing large datasets.
  • Aggregated Temperature Statistics - Reads a large text file of weather station temperature readings and computes the min, mean, and max per station.
  • Memory-Mapped File Access - Reads the input file by mapping it directly into virtual memory, avoiding traditional buffered reads for faster access.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI
  • Grouped Aggregations - Computing summary statistics like min, mean, and max across grouped data records.
  • Primitive - Uses a hand-optimised hash map with primitive long keys and int values to minimise object overhead and garbage collection.
  • Chunked File Processing - Splits the file into chunks processed concurrently by multiple threads, aggregating partial results before merging.
  • Text File Processing Benchmarks - A benchmark that tests how efficiently a Java program can parse and aggregate structured data from a plain text file.
  • Branch-Less Parsing Techniques - Parses temperature values using bitwise operations and integer arithmetic instead of branching, reducing CPU pipeline stalls.
  • Zero-Allocation Architectures - Pre-allocates all working data structures upfront and avoids object creation during the hot loop to eliminate GC pauses.
  • Thread-Local Aggregation - Assigns each processing thread its own aggregation map to eliminate lock contention, merging results only at the end.
  • Java Benchmarking Tools - A tool for measuring and optimizing Java code execution speed against a fixed data processing challenge.
  • Large File Processing - Reading and aggregating data from text files with billions of rows efficiently.
  • CPU Profilers - Identifying performance bottlenecks in Java code using flamegraphs and execution time analysis.
  • Direct Memory Access - Leverages Unsafe for direct memory operations on the mapped file, bypassing bounds checks for maximum throughput.
  • JIT-Friendly Loop Unrolling - Writes tight, manually unrolled loops that the JIT compiler can further optimise into efficient native machine code.

    • Star history

    Star history chart for gunnarmorling/1brcStar history chart for gunnarmorling/1brc

    How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.

    Frequently asked questions

    What does gunnarmorling/1brc do?

    The 1BRC (One Billion Row Challenge) is a Java performance benchmarking exercise that processes one billion temperature records from a text file to compute the minimum, mean, and maximum temperature per weather station. At its core, it is a large-scale data aggregation challenge designed to test how efficiently a Java program can parse and aggregate structured data from a plain text file, serving as both a programming exercise and a benchmark for Java performance…

    What are the main features of gunnarmorling/1brc?

    The main features of gunnarmorling/1brc are: Data Aggregation Challenges, Performance Benchmarks, Aggregated Temperature Statistics, Memory-Mapped File Access, Grouped Aggregations, Primitive, Chunked File Processing, Text File Processing Benchmarks.

    What are some open-source alternatives to gunnarmorling/1brc?

    Open-source alternatives to gunnarmorling/1brc include: openhft/chronicle-queue — Chronicle Queue is a high-performance data handling system featuring off-heap message queues, memory-mapped file… rkyv/rkyv — rkyv is a zero-copy deserialization framework for Rust that provides a binary serialization format for memory-mappable… rdatatable/data.table — This project is a high-performance tabular data processing framework for R, designed to handle massive datasets with… mstange/samply — Samply is a cross-platform CPU sampling profiler and performance analysis utility. It consists of a command-line tool… kdab/hotspot — Hotspot is a graphical user interface for analyzing and visualizing performance data captured by the Linux perf tool.… prestodb/presto — Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data…

    Open-source alternatives to 1brc

    Similar open-source projects, ranked by how many features they share with 1brc.
    • openhft/chronicle-queueOpenHFT avatar

      OpenHFT/Chronicle-Queue

      3,692View on GitHub↗

      Chronicle Queue is a high-performance data handling system featuring off-heap message queues, memory-mapped file stores, and replicated message stores. It provides a binary compatible memory layout that enables different programming languages to share data without serialization overhead. The system utilizes a replicated message store to synchronize data across multiple nodes, ensuring high availability and instant failover. Its memory-mapped architecture supports deterministic replay from disk and low-latency data recording. The project implements off-heap memory management and zero-allocati

      Javachroniclejavalow-latency
      View on GitHub↗3,692
    • rkyv/rkyvrkyv avatar

      rkyv/rkyv

      4,267View on GitHub↗

      rkyv is a zero-copy deserialization framework for Rust that provides a binary serialization format for memory-mappable data archives. It allows complex data structures to be mapped to bytes and accessed directly from a buffer without allocating new memory or copying data. The project enables the serialization of polymorphic types and trait objects, maintaining their dynamic behavior and structure within the binary form. It utilizes relative-pointer addressing and byte-aligned structure packing to ensure data remains valid regardless of where it is loaded in memory. The framework covers high-

      Rustrustserializationzero-copy
      View on GitHub↗4,267
    • rdatatable/data.tableRdatatable avatar

      Rdatatable/data.table

      3,894View on GitHub↗

      This project is a high-performance tabular data processing framework for R, designed to handle massive datasets with memory efficiency and speed. It provides an enhanced data structure that utilizes reference semantics and in-place modification to perform complex transformations without the overhead of unnecessary object copying. The library distinguishes itself through its low-level architectural optimizations, including multi-threaded parallel processing, radix-based sorting, and memory-mapped file parsing. By offloading critical data manipulation and aggregation routines to compiled C code

      R
      View on GitHub↗3,894
    • mstange/samplymstange avatar

      mstange/samply

      4,263View on GitHub↗

      Samply is a cross-platform CPU sampling profiler and performance analysis utility. It consists of a command-line tool for recording process stack traces at regular intervals and a visual interface for analyzing the resulting execution data. The system includes a debug symbol resolver that maps raw memory addresses to human-readable function names using local or remote symbol information. It transforms recorded execution data into flame graphs and timelines to pinpoint function-level hotspots. The tool provides capabilities for CPU execution recording, stack unwinding, and symbol resolution a

      Rust
      View on GitHub↗4,263
    See all 30 alternatives to 1brc→