BurntSushi/xsvArchived
Xsv
xsv is a suite of high-performance command-line utilities written in Rust for the analysis, manipulation, and statistical processing of large delimited datasets. It provides a toolkit for processing comma-separated value files through a command line interface.
The project provides capabilities for statistical analysis, including the computation of column statistics, value frequencies, and descriptive metrics. It also includes data manipulation utilities for joining, slicing, sampling, and reformatting records.
The toolkit covers a broad range of data operations including column selection, data filtering, and structural modifications such as concatenating or splitting records. It further supports dataset indexing for faster row access and visual rendering of data as aligned tables.
Features
- Command-Line Data Processors - Provides a comprehensive suite of high-performance Rust-based command-line tools for processing large CSV datasets.
- CSV Command Line Toolkits - A high-performance suite of command-line tools for analyzing, filtering, and manipulating CSV data files.
- CSV Processing - Provides a comprehensive toolkit for joining, slicing, sampling, and reformatting CSV records.
- Column Manipulations - Allows extracting or re-ordering specific columns to refine the dataset output.
- CSV Record Filters - Isolates CSV records using regular expressions and specific column value matching.
- Data Joins - Provides high-performance inner, outer, and cross joins for combining large CSV datasets.
- Frequency Analyzers - Builds frequency tables for each column to determine how often specific values occur.
- Row-Based Result Streaming - Processes records sequentially through a pipeline to maintain a low memory footprint for massive datasets.
- Single-Pass Column Analyzers - Enables efficient computation of descriptive statistics and value frequencies using a single-pass scan of the dataset.
- Statistical Analysis Libraries - Calculates column statistics, value frequencies, and other descriptive metrics from CSV datasets.
- CSV Descriptive Statistics - Calculates mean, median, standard deviation, and range for every data column in a CSV.
- CSV Format Converters - Changes delimiters, record terminators, and quoting rules to ensure compatibility with other tools.
- CSV Record Manipulators - Provides operations for concatenating, splitting, reversing, or padding CSV records.
- CSV Row Indexes - Generates metadata indexes for CSV files to enable constant-time row access.
- CSV Row Slicers - Extracts specific ranges of CSV rows using indices to avoid parsing the entire dataset.
- Reservoir Samplers - Implements reservoir sampling to extract random subsets of data while keeping memory usage constant.
- Offset-Based Addressing - Uses byte-offset indexing to provide constant-time random access to rows without parsing the entire file.
- CSV and Tabular Data - Fast CSV indexing, slicing, and analysis.
- Tabular Data Processing - High-performance CSV toolkit written in Rust.
- Terminal Productivity - Fast CSV command-line toolkit.
- Text Processing and Search - Fast CSV command line toolkit.
