lemire/simdjson
Simdjson
simdjson is a high-performance JSON parser that utilizes SIMD instructions to process gigabytes of data per second. It functions as a SIMD JSON parser, a multithreaded NDJSON processing library, a UTF-8 validation engine, and a tool for JSON minification and string building.
The project focuses on high-throughput data processing, enabling the ingestion of massive JSON volumes and the verification of UTF-8 encoding standards. It includes dedicated capabilities for constructing JSON strings with optimized memory usage and removing unnecessary whitespace from documents to reduce file size.
The system supports the processing of newline-delimited JSON files through multithreaded functions and provides a specialized builder for efficient JSON generation.
Features
- SIMD Parsers - Uses SIMD vector instructions to process multiple bytes simultaneously for high-speed structural analysis of JSON.
- High-Performance Parsers - Implements a JSON parsing library optimized for extreme throughput using SIMD and hardware-specific acceleration.
- JSON Parsers - Implements a high-performance library for parsing and mapping JSON data structures.
- JSON Processing - Provides high-speed serialization and deserialization of JSON data using specialized hardware instructions.
- NDJSON Parsers - Provides a high-performance multithreaded parser specifically optimized for newline-delimited JSON data.
- Encoding Validators - Provides a high-throughput engine to verify that documents adhere to the UTF-8 encoding standard.
- Big Data Processing - Accelerates the ingestion of massive NDJSON volumes into analytics engines via multithreaded processing.
- JSON Construction Utilities - Provides programmatic JSON construction utilities to optimize memory usage and performance.
- JSON Minifiers - Removes unnecessary whitespace from JSON documents to reduce file size while preserving data.
- JSON Content Generators - Programmatically constructs JSON strings using a specialized builder to optimize memory usage.
- Branchless Indexing - Implements branchless index generation to avoid conditional jumps during JSON tokenization.
- Lazy Materialization - Delays the conversion of raw bytes into typed data until the specific field is requested.
- SIMD Buffer Padding - Allocates extra space at the end of buffers to allow wide SIMD loads without memory access violations.
- Two-Stage Parsing - Separates initial structural scanning from final value extraction to maximize CPU pipeline efficiency.
- Procesare JSON - Extremely fast SIMD-accelerated JSON parsing.
- Serialization and Parsing - SIMD-accelerated JSON parsing library.
- Json - Listed in the “Json” section of the Awesome Cpp Cn awesome list.
