This is a set of command line utilities for manipulating large tabular data files. Files of numeric and text data commonly found in machine learning and data mining environments. Filtering, sampling, statistics, joins, and more.
The main features of ebay/tsv-utils are: CSV and Tabular Data, Text Processing.
Open-source alternatives to ebay/tsv-utils include: google/re2 — re2 is a C++ regular expression library designed for high-performance text processing. It is a non-backtracking regex… lancopku/pkuseg-python — pkuseg-python is a Chinese word segmentation toolkit and natural language processing library. It provides specialized… azaitsev/millify. carpedm20/emoji — emoji terminal output for Python. chardet/chardet — Python character encoding detector. burntsushi/xsv — xsv is a suite of high-performance command-line utilities written in Rust for the analysis, manipulation, and…
re2 is a C++ regular expression library designed for high-performance text processing. It is a non-backtracking regex engine that provides linear-time pattern matching, ensuring that execution time remains proportional to the size of the input string regardless of the pattern used. The library supports UTF-8 and Latin-1 text encodings for searching and extracting substrings. It includes capabilities for multi-pattern optimization, allowing multiple regular expressions to be combined into a single representation to scan text for several patterns in one pass. The project covers core regex oper
pkuseg-python is a Chinese word segmentation toolkit and natural language processing library. It provides specialized models for splitting Chinese text into words across various domains, including news, medical, and web content, and includes a tool for assigning grammatical parts of speech tags to segmented words. The library allows for the training of custom segmentation models using annotated datasets and supports the integration of user-defined dictionaries to ensure specialized terminology is recognized correctly. It employs a multi-threaded execution engine to process large volumes of Ch
xsv is a suite of high-performance command-line utilities written in Rust for the analysis, manipulation, and statistical processing of large delimited datasets. It provides a toolkit for processing comma-separated value files through a command line interface. The project provides capabilities for statistical analysis, including the computation of column statistics, value frequencies, and descriptive metrics. It also includes data manipulation utilities for joining, slicing, sampling, and reformatting records. The toolkit covers a broad range of data operations including column selection, da