30 open-source projects similar to wader/fq, ranked by shared indexed features. Tags may describe platforms or build tools rather than the same primary purpose. Check each project’s use case, license, and deployment requirements before treating it as a replacement.
Wireshark is a network protocol analyzer and traffic inspector used for capturing and inspecting network traffic. It functions as a packet capture tool that intercepts live data from network interfaces and a TCP/IP dissector that decodes network protocol layers to translate raw binary packets into human-readable fields. The system provides capabilities for protocol stream reconstruction, grouping related packets into cohesive conversations between endpoints. It also operates as a packet file converter, allowing for the reading, modification, and conversion of network capture files across vari
Angle Grinder is a command line log processor and analytics tool used for parsing, filtering, and aggregating logs through a pipeline of text transformations. It functions as a text transformation pipeline that converts unstructured logs, as well as JSON and logfmt serialized data, into structured fields for analysis. The tool enables the computation of summary statistics, including running totals, counts, averages, and percentiles. It specifically supports time series log processing by partitioning data into discrete time windows to analyze event frequency and system behavior. The processin
Miller is a command-line data processor used for filtering, transforming, and aggregating name-indexed tabular data. It functions as a tool for querying and reshaping records across multiple file formats, serving as a converter between CSV, JSON, and YAML. The tool distinguishes itself by using a name-indexed data model, allowing users to manipulate fields by name rather than numeric position. It utilizes single-pass streaming algorithms to compute statistics and summaries on large datasets that exceed available system memory. Its capabilities cover data transformation and analysis, includin
MessagePack-CSharp is a high-performance binary serialization library for .NET applications that converts object graphs into the MessagePack format. It functions as a C# data serialization toolkit and a polymorphic binary encoder capable of handling abstract classes and interfaces using union keys to identify concrete derived types. The library provides a binary format transcoder to transform binary data into human-readable JSON for debugging. It supports ahead-of-time formatter generation to avoid runtime overhead and implements LZ4 binary compression to reduce the size of serialized data.
Clojure is a general-purpose, functional programming language hosted on the Java Virtual Machine. It is a homoiconic S-expression language that represents programs as nested data structures, allowing code to be manipulated and evaluated as data. The project provides a framework for JVM interoperability, enabling the invocation of Java methods and integration with other JVM-based languages. It distinguishes itself through a persistent data structure library that uses bitmapped vector tries to manage immutable collections and a programmatic REPL for interactive software development and real-tim
Zeek is a network analysis framework and security monitoring tool that transforms raw network packets into high-level semantic logs. It functions as an application protocol analyzer and network intrusion detection system designed to extract meaning from network traffic and monitor for malicious activity. The system focuses on archiving network activity and maintaining historical records of application-layer state for forensic investigation and auditing. It utilizes a combination of modular protocol analyzers and customizable detection policies to perform deep semantic analysis of numerous app
React Native Debugger is a standalone developer application for inspecting and debugging JavaScript runtimes in mobile applications built with React Native. It provides a dedicated interface to monitor the runtime state, network requests, and console logs of a remote mobile environment. The tool integrates specialized inspectors for Redux state management and GraphQL client debugging, allowing for time-traveling state transitions and the analysis of queries, mutations, and local cache. It also features a component hierarchy viewer for visualizing and modifying UI properties in real time. Bro
Tproxy is a set of command-line utilities designed for throttling bandwidth, simulating network latency, proxying TCP traffic, and analyzing connection stability. It provides a toolset for redirecting network traffic from local ports to remote destinations while monitoring the resulting connection behavior. The project differentiates itself by combining traffic manipulation with diagnostic monitoring. It can introduce configurable packet delays and cap upload and download transfer speeds to simulate constrained or low-bandwidth network environments. Beyond traffic control, the utilities incl
This project is a comprehensive repository of fundamental computer science algorithms and data structures designed as a reference for academic study, technical interview preparation, and competitive programming. It provides standardized implementations of core computational strategies, serving as an educational resource for developers to master software engineering fundamentals and algorithmic problem-solving. The collection distinguishes itself through a multi-language approach, offering cross-language solutions for complex tasks ranging from graph traversal and dynamic programming to bitwis
Tengo is a dynamic, embeddable scripting language for Go applications that allows for the execution of custom scripts without requiring the host binary to be recompiled. It operates as a bytecode-compiled virtual machine, transforming source code into a compact intermediate representation for execution on a stack-based engine. The system is designed as a secure scripting sandbox, enforcing strict limits on memory allocation and execution time to safely run untrusted code. It supports concurrent script execution by cloning compiled bytecode and using recursive immutability conversion to share
ImHex is a professional-grade hex editor and binary data analysis platform designed for inspecting, modifying, and reverse engineering raw file contents. It functions as a schema-driven engine that interprets complex binary structures by applying custom definitions to map and visualize byte-level data. The platform distinguishes itself through a dedicated domain-specific language that allows users to define structural schemas for automated file parsing. This capability is supported by a dynamic plugin architecture and an event-driven registry, which enable the integration of external modules
python-magic is a C-binding wrapper that provides a Python interface for the libmagic system library. It functions as a file signature analyzer and MIME type detector, identifying file formats by comparing header bytes against a database of known binary signatures. The library enables the identification of file types from both file paths and raw data buffers. It supports custom file signature matching through the injection of user-provided magic databases, allowing for the detection of specialized or proprietary formats. The project covers binary data analysis and MIME type mapping to transl
This tool is a command-line processor designed for querying, updating, and transforming structured data files. It functions as a versatile engine for manipulating YAML, JSON, TOML, and XML documents, allowing users to perform complex operations directly from the terminal. By utilizing a path-based expression language, it enables precise navigation and modification of data structures within configuration files and infrastructure-as-code workflows. What distinguishes this tool is its ability to perform in-place document mutations while preserving original formatting, comments, and metadata. It
jq is a command-line JSON processor and data transformer. It provides a functional query language used to slice, filter, map, and transform structured JSON data directly within a terminal. The utility functions as a data transformer that reshapes JSON input into different structures or formats based on declarative logic. This allows for the extraction and analysis of structured data from sources such as API responses and system logs.
Hexyl is a colored hex dump utility and binary data viewer for the terminal. It allows for the inspection of binary files by rendering contents as a colored hex dump to distinguish between different byte categories, such as printable text, whitespace, and null bytes. The tool includes a C-style hex exporter that transforms binary data into C include files for direct integration into source code. It supports visual layout customization through configurable panels and borders, as well as the ability to define colors for byte categories and offsets using terminal colors or RGB hex codes via envi
Polars is a high-performance columnar data processing library designed for efficient analytical workflows. It functions as a structured data library that organizes information into typed columns, utilizing the Apache Arrow memory format to enable zero-copy data sharing and cache-friendly, vectorized operations. The engine is built to handle large-scale tabular datasets, providing both local and distributed analytical runtimes that scale from single-machine environments to multi-node clusters. The project distinguishes itself through a sophisticated lazy query engine that constructs abstract e
This project is a command-line processor designed for the parsing, filtering, and transformation of structured data streams. It functions as a declarative programming environment that treats data as immutable streams, allowing users to perform complex structural modifications through the composition of small, reusable functions. By utilizing a recursive tree traversal engine, the system enables the navigation, inspection, and modification of deeply nested hierarchical data structures. The engine distinguishes itself through a stream-oriented architecture that processes input records one by on
Fx is a command-line processing suite designed for the transformation, conversion, exploration, and visualization of structured data. It functions as a terminal-based utility that handles both automated shell pipelines and interactive navigation of complex, nested data hierarchies. The tool distinguishes itself by integrating a JavaScript-based engine that executes user-provided logic to filter, map, or modify data fields within a sandboxed runtime. It maintains a responsive interface by decoupling data processing from the display loop, allowing users to explore large datasets through an inte
Binwalk is a firmware analysis and reverse engineering tool designed to identify, extract, and analyze embedded files and data types within binary firmware blobs. It functions as a binary file signature scanner and entropy analyzer to decompose firmware images into their constituent components. The tool distinguishes itself by combining signature-based pattern matching for known binary headers with entropy analysis. By calculating data randomness across file offsets, it can locate compressed or encrypted sections that do not possess known signatures. The project covers binary data forensics
Skill Seekers is a toolset for generating large language model knowledge bases, featuring a multi-source content scraper and a dedicated RAG data pipeline. It extracts technical data from documentation, code, and video to create structured assets and configuration files for AI-powered IDE extensions. The project distinguishes itself through the ability to transform raw data into polished tutorials and specialized skills for AI plugin marketplaces. It utilizes abstract syntax tree parsing and optical character recognition to analyze GitHub repositories, PDFs, and video frames, converting these
Apache DataFusion is an extensible, columnar SQL query engine that runs embedded within a host application without requiring a separate server process. It processes data in columnar batches using Apache Arrow for memory-efficient analytics, and can scale analytic workloads across multiple nodes for parallel execution. The engine supports both SQL and DataFrame queries through a modular, streaming architecture that allows custom operators, data sources, functions, and optimizer rules. The engine distinguishes itself through its modular extension framework, which enables building custom query e
CloudEvents is an open specification for describing event data in a common format across cloud platforms and services. It defines a standard structure and set of metadata attributes for events, enabling interoperability across different systems so producers and consumers can exchange events without custom translation. The specification provides a protocol-agnostic serialization framework that maps CloudEvents attributes and payloads to multiple serialization formats including JSON, Avro, and Protobuf, and defines transport bindings for mapping events onto protocols like HTTP, AMQP, Kafka, MQTT
Nanopb is a lightweight C implementation of Protocol Buffers designed for resource-constrained systems and microcontrollers. It functions as both an embedded serialization library for encoding and decoding structured data and a code generator that transforms definition files into compact C source and header files. The project is distinguished by its focus on static memory allocation, using fixed maximum sizes for strings and arrays to avoid dynamic memory allocation and heap fragmentation. It employs a callback-based streaming mechanism to process messages or fields that exceed available phys
nom is a parser combinator framework for Rust used to build complex parsers by combining small, reusable parsing functions. It functions as a zero-copy parsing tool that minimizes memory overhead by returning slices of the original input instead of allocating new memory. The framework is designed for diverse data formats, serving as a binary data parser with configurable endianness and a bitstream processing library capable of extracting values of arbitrary bit length. It also functions as a streaming data parser that can process data arriving in chunks and signal when additional input is req
nom is a Rust parser combinator framework used to build complex parsers for binary and text data. It functions as an abstract syntax tree generator and a bit-level binary parser, allowing users to construct structured data by combining small, reusable parsing functions. The framework provides specialized support for zero-copy binary parsing, extracting data as slices from raw byte arrays to avoid memory allocations. It also includes a streaming data parser capable of processing partial input chunks from networks or files and signaling when additional input is required. The project covers a b
OpenDBC is a CAN bus interface library and telemetry parser designed to read and write messages to a vehicle network. It functions as an automotive hardware abstraction layer that maps generic control signals to model-specific bus IDs across different vehicle brands, while acting as a control system that converts high-level steering and acceleration requests into raw binary packets for vehicle actuators. The project includes a vehicle network safety gateway that restricts or blocks specific control messages based on the system state to prevent unsafe maneuvers. It provides a standardized way
This is a structured deep learning curriculum for programmers, delivered as a collection of Jupyter notebooks. It teaches the fundamentals of training neural networks for computer vision, natural language processing, tabular data analysis, and collaborative filtering using PyTorch and the fastai library. The course is designed to be hands-on, guiding learners from building a training loop from scratch to fine-tuning pretrained models for a variety of practical tasks. The curriculum distinguishes itself by covering the full lifecycle of a deep learning project, from data preparation and augmen
This project is a reverse engineering implementation and library designed to parse and decrypt the binary data streams used by the WhatsApp web client. It provides a WebSocket client library and tools to communicate with messaging servers by implementing reverse engineered protocol patterns. The system enables the development of custom clients and automation tools through the emulation of browser sessions and the management of persistent binary streams. It handles account authentication via QR code generation and pairing strings to link devices and maintain encrypted sessions. The project co
dnSpy is a specialized suite of tools for the reverse engineering of .NET assemblies, functioning as a decompiler, assembly editor, and debugger. It translates compiled intermediate language back into high-level source code and provides an execution environment for stepping through compiled binaries to inspect runtime state without the original source files. The project includes a BAML decompiler that converts binary application markup language into a disassembled format and translates it into extensible markup language for user interface analysis. It also functions as a binary analysis tool