phiresky/ripgrep-all
Ripgrep All
ripgrep-all is a command-line utility that extends ripgrep to perform regular expression searches across binary files, compressed archives, and media formats. It functions as a universal text extractor that converts non-plain-text formats, such as PDFs, E-books, and Office documents, into searchable text.
The tool uses a system of adapters to transform binary data into plain text and utilizes a local database to cache these extracted versions, accelerating repeated search operations. It identifies file types by analyzing header magic bytes rather than relying on file extensions.
The project covers recursive archive traversal for searching inside nested zip and tar files, as well as the retrieval of media metadata, subtitles, and chapters from video and audio containers. It also provides a mechanism to integrate custom extraction scripts for proprietary file formats.
Features
- Text Pattern Search - Performs high-performance regular expression searches across a mix of plain text, PDFs, E-books, and Office documents.
- Binary Format Text Extraction - Provides regular expression searching across binary formats like PDFs and Office documents by extracting them into plain text.
- Compressed Data Searching - Enables regular expression searching directly within compressed archives by analyzing internal mime types.
- Persistent Binary Caches - Utilizes a local database to cache extracted text from binary files, preventing redundant processing.
- Binary - Converts binary documents into searchable formats and indexes them for high-performance querying.
- Regex-Based File Search - Extends ripgrep's regex search capabilities to include binary files like PDFs and Office documents.
- Text Extraction - Provides utilities to extract plain text from various binary document formats for search processing.
- Recursive Archive Traversers - Recursively descends into nested archives to identify and search internal contents based on mime types.
- Magic Byte File Identification - Identifies the correct text extractor by analyzing file header magic bytes instead of relying on file extensions.
- Text Extraction Caches - Maintains a local database of extracted text from binary files to speed up repeated search operations.
- Universal Text Extractors - Implements a system of adapters to convert various binary formats into searchable plain text.
- Archive Content Scanning - Scans the contents of compressed archives like zip and tar files without requiring manual extraction.
- Signature-to-Adapter Mappings - Implements a flexible mapping system that links file signatures to specific extraction tools via configuration.
- Media Metadata Retrievers - Retrieves and searches embedded subtitles, chapters, and metadata from video and audio containers.
- Subprocess Management - Orchestrates external CLI tools via subprocesses to normalize diverse binary formats into searchable text streams.
- Content-Based Metadata Extraction - Retrieves and searches embedded subtitles, lyrics, and chapter titles within video and audio files.
- Custom Format Decoders - Allows the integration of custom scripts and external tools as adapters for proprietary file formats.
- Text Processing - Search tool for text within PDFs, archives, and documents.
