Projects sharing features with Data Hacks

Presto is a distributed SQL query engine designed for high-performance analytical processing across heterogeneous data sources. It functions as a data federation platform and massively parallel processing engine, allowing users to execute interactive queries against diverse storage systems without requiring data migration. By mapping remote metadata and structures to a unified relational namespace, it enables seamless cross-platform analysis through a standard SQL interface. The engine distinguishes itself through a pluggable connector architecture and a shared-nothing distributed processing

Termgraph is a terminal data visualization library and command line analytics tool used to render bar charts, histograms, and heatmaps directly in the shell. It utilizes ANSI escape sequences and Unicode characters to generate colorful visual data representations within a text-based environment. The tool provides specialized capabilities for transforming raw datasets into horizontal or vertical bar graphs, column charts, and stacked charts. It also functions as a heatmap generator, mapping time-series data to a calendar layout to visualize temporal patterns over a year. The library supports

csvkit is a composable Unix-style command-line toolkit for converting, filtering, and analyzing CSV files directly from the terminal. It provides a suite of focused single-purpose commands that can be combined via pipes to build complex data processing workflows, with a modular architecture that includes a column-type inference engine for automatically detecting data types and a streaming-pipeline design for efficient handling of tabular data. The toolkit distinguishes itself through its SQL-engine abstraction layer, which allows users to run SQL queries directly against CSV files without req

qsv is a high-performance command line toolkit for querying, transforming, and analyzing comma-separated value files. It functions as a data wrangling interface and a tabular data profiler, featuring a query engine capable of executing SQL statements and joins directly on flat files without requiring a database. The project is distinguished by its ability to process massive datasets that exceed available system memory. This is achieved through disk-based external memory processing, including multithreaded merge sorting, on-disk hash tables for deduplication, and lightweight file indexing for

Xan is a command-line tool and data transformation engine for processing CSV, TSV, and JSONL datasets. It functions as a processor for compressed files, enabling random access and seeking within gzipped and Zstd files, and serves as a converter for specialized bioinformatics data formats. The tool handles large datasets without requiring full memory loads by utilizing stream-based processing. It provides capabilities for merging, sorting, and deduplicating massive files, as well as converting data between various tabular formats. The project covers a broad range of data wrangling and analysi

Clip is a command-line data visualization tool designed to generate image-based charts and diagrams from text descriptions. It functions as a chart generator that converts written data and descriptive patterns into visual formats without the use of a graphical user interface. The tool specializes in producing scalable vector graphics, translating text-to-chart transformations into XML-based vector paths. This approach allows for the automated creation of technical illustrations and diagrams specifically suited for developer documentation. The system employs a template-driven layout engine to

Csvlens is a terminal-based tabular data explorer and memory-mapped browser designed for viewing and navigating large CSV files. It functions as a command-line interface for inspecting massive structured datasets while maintaining a constant memory footprint. The tool provides a text-based user interface for exploring data, featuring a custom layout engine that supports column resizing, line wrapping, and frozen headers. It includes a search mechanism to locate and visually highlight text patterns using regular expressions. Capabilities cover data manipulation through alphabetical or natural

This project is a comprehensive educational resource and technical guide for Bash shell programming and command-line operations. It serves as a programming guide, scripting reference, and tutorial for navigating Unix-like terminal environments. The documentation covers a broad range of system administration and automation tasks, including remote server administration via secure shell connections and the management of system processes and resources. It provides detailed instructions on executing remote commands and performing secure file transfers between hosts. The guide details core scripti

Streamlink is a command line video stream extractor that retrieves direct stream URLs from online services for use in external media players. It functions as a local media stream pipe, redirecting raw video data from web services into local files or players via standard input or HTTP. The project includes a headless browser stream scraper to intercept network requests and extract media data from script-heavy websites, alongside a dedicated processor for HLS and DASH segmented media streams. The tool utilizes a modular video plugin framework, allowing support for new streaming platforms to be

Zsh is a Z Shell implementation that provides an interactive command-line interface for executing system commands, managing files, and automating workflows. It serves as a POSIX-compliant shell and a programmable scripting environment for writing automation scripts. The project focuses on providing a Unix shell environment with a high degree of customization. It includes advanced autocomplete, programmable completion, and sophisticated globbing to tailor terminal behavior and environment preferences. The system covers command-line workflow automation and the execution of system commands thro

Viddy is an interactive terminal command monitor and watcher that executes shell commands at recurring intervals to track changes in output over time. It functions as a command execution history pager and output difference viewer, allowing users to monitor real-time state changes within a terminal interface. The tool distinguishes itself by providing playback and history navigation capabilities, enabling users to move backward and forward through a recorded sequence of previous command outputs. It includes a dedicated output diff viewer that visually highlights text additions or removals betw

Bash-Oneliner is a curated collection of reusable shell snippets and command-line patterns designed for system administration and data processing in Unix-like environments. It serves as a productivity guide for executing efficient terminal operations, text stream manipulation, and routine maintenance tasks using native shell primitives. The project focuses on modular command composition, allowing users to build complex workflows by chaining standard utilities through pipe-based data streaming. It emphasizes the use of POSIX-compliant shell execution and regular expression-powered text process

This tool is a command-line processor designed for querying, updating, and transforming structured data files. It functions as a versatile engine for manipulating YAML, JSON, TOML, and XML documents, allowing users to perform complex operations directly from the terminal. By utilizing a path-based expression language, it enables precise navigation and modification of data structures within configuration files and infrastructure-as-code workflows. What distinguishes this tool is its ability to perform in-place document mutations while preserving original formatting, comments, and metadata. It

This project is a command-line video downloader and web media extractor written in Python. It is designed to retrieve video and audio streams from various hosting platforms for local storage or real-time streaming via standard output. The system utilizes a framework of custom extractor classes to handle different websites and allows for the development of new extractors to extend compatibility. It supports accessing restricted, private, or region-locked content through the use of session cookies, user-agent headers, and proxy server routing. Capabilities include media format selection based

bqplot is an interactive data visualization library for IPython and Jupyter notebooks that utilizes a grammar of graphics. It functions as a tool for creating 2D charts and maps with real-time updates and bidirectional communication between the kernel and frontend. The library is distinguished by its ability to act as a geographic data visualization tool, rendering choropleth maps and spatial data via GeoJSON and custom projections. It also serves as a financial charting tool for producing OHLC and candle bar charts, and as an interactive dashboard framework for combining plotting widgets wit

Spark is a suite of command line utilities designed to render numeric data sets as compact visual graphs using Unicode characters. It functions as a terminal data visualizer and charting tool that converts sequences of numbers into visual trend lines within a shell environment. The tool generates sparkline graphs by mapping numeric value ranges to specific Unicode block characters. It utilizes linear value scaling to normalize input numbers to fit within available character heights and processes data sequentially from the standard input stream to produce real-time visual output. These capabi

Failsafe is a JVM resilience library providing a collection of fault tolerance patterns for Java applications. It functions as an asynchronous execution wrapper that runs tasks in the background and returns futures to prevent thread blocking. The library allows for the composition of resilience policies, enabling multiple patterns to be stacked into a sequential pipeline. It includes specific implementations for circuit breaking to prevent system overload, rate limiting to control traffic flow, and a framework for managing retries and fallbacks. Capability areas cover traffic management thro

rich-cli is a terminal rich text renderer and data visualizer that transforms raw text and structured data into styled console output. It functions as a command line markup processor, converting Markdown and custom markup into formatted text with clickable links and colors. The tool specializes in terminal content rendering for various formats, including syntax-highlighted source code, JSON data with consistent indentation, and CSV tables. It also supports the rendering of Jupyter notebooks and provides a terminal content pager for navigating long files and remote content via a scrollable int

Tbox is a comprehensive collection of low-level utility libraries and toolkits for portable C development. It provides a multi-platform C utility library that serves as a lightweight replacement for standard libraries, offering optimized memory management and system abstractions to ensure hardware compatibility across different operating systems. The project distinguishes itself through a specialized framework of standardized data containers, including hash tables, vectors, and linked lists with integrated searching and sorting. It further provides a dedicated toolkit for networking and seria

Croc is a command-line utility for sending files and folders between computers using end-to-end encrypted peer-to-peer connections. It employs elliptic curve encryption and key agreement to secure data transmission between remote endpoints. The tool allows users to coordinate transfers using a shared code phrase and supports the operation of custom relay servers to facilitate connections without relying on public infrastructure. It also includes a proxy client to route encrypted traffic through SOCKS5 proxies. Additional capabilities include resumable data transmission for unstable connectio

tbox is a multi-platform C standard library and toolkit designed for portable software development. It provides a core foundation for low-level system programming through a collection of system primitives, data structures, and memory management tools. The project includes a coroutine framework for cooperative multitasking and asynchronous logic, a data structure library featuring generic containers with a consistent iterator interface, and a memory management toolkit for pool allocation and debugging of heap overflows and leaks. It also provides a network and serialization toolset for handlin

This project is a curated directory of command line applications and utilities designed to enhance developer productivity and streamline technical workflows. It serves as a comprehensive index of open-source software, categorizing tools that assist with system administration, development automation, and personal task management. The repository distinguishes itself by providing a structured collection of terminal-based software that spans diverse functional domains. It includes resources for managing infrastructure and cloud resources, performing code maintenance, and customizing terminal envi

Fabric is a command-line interface and framework designed to integrate artificial intelligence reasoning into shell-based workflows. It functions as an orchestration tool that connects local data pipelines to remote artificial intelligence services, allowing users to automate content analysis and complex reasoning tasks directly from the terminal. The project distinguishes itself through a modular architecture that treats prompt patterns as version-controlled, reusable logic stored on the local filesystem. By utilizing standard input and output streams, it enables users to chain these analyti

This project serves as a comprehensive reference and toolkit for Linux command-line operations. It provides a structured collection of Bash commands and terminal utilities designed to assist with system administration, file management, and process control within POSIX-compliant environments. The resource functions as a guide for mastering terminal productivity and shell scripting. It covers essential workflows including efficient filesystem navigation, command history management, and the application of shell operators for task automation. Users can utilize these materials to refine their inte

TextMate is a programmable text editor designed for software development and project management. It functions as a highly customizable environment where users can define language-specific behaviors, syntax highlighting rules, and automated workflows to suit their individual development needs. The editor distinguishes itself through a modular, bundle-based extensibility model that allows for deep integration with system shell commands. By piping document buffers through external scripts and command-line tools, users can perform complex text transformations, automate file lifecycle tasks, and b

This project is a high-performance command-line utility designed for rapid filesystem navigation and file discovery. It enables users to locate files and directories within large project structures using recursive search, pattern matching, and metadata-aware filtering. By employing multi-threaded parallel traversal, it provides an efficient way to explore complex directory trees. What distinguishes this tool is its ability to integrate directly into terminal workflows and automate file management tasks. It automatically respects version control ignore files and hidden file settings, ensuring

FinceptTerminal is a quantitative finance platform and financial engineering library designed for asset valuation, risk management, and fixed-income analytics. It provides a comprehensive suite for algorithmic trading and investment strategy automation, integrating specialized language model agents and node-based workflows to automate market research and alpha generation. The project distinguishes itself with a dedicated game theory analysis engine for calculating Nash equilibria and simulating strategic interactions in competitive markets. It also features a specialized credit risk modeling