apache/tika
Tika
Tika is a content analysis toolkit and Java library designed for detecting and extracting metadata and text from thousands of different file types. It functions as a universal document text extractor and metadata extraction engine, converting complex files into plain text or XHTML.
The system employs a specialized MIME type detector that identifies document formats using magic bytes and metadata to determine the correct parser. It serves as an OCR integration gateway, connecting to external text recognition tools to extract content from image files.
The project covers a broad range of extraction and analysis capabilities, including digital asset metadata retrieval, email archive processing for formats like PST and mbox, and natural language detection. It further supports automated document parsing, recursive archive unpacking, and text content analysis through integrations for sentiment classification and named entity recognition.
Features
- Content Extraction - Provides a unified interface for retrieving raw text and metadata from a vast variety of document types.
- MIME Type Detection Engines - Determines file formats by analyzing magic bytes, filenames, and metadata using a sequence of prioritized detectors.
- Document Text Extractors - Converts complex binary files like PDFs and Office documents into plain text or XHTML for downstream processing.
- Stream-Based Parsing - Implements event-driven parsing of large documents to extract text while minimizing memory consumption.
- General Metadata Engines - Implements a framework for retrieving embedded properties and descriptive attributes from diverse media and document formats.
- Content Analysis Toolkits - Ships a comprehensive Java library for detecting and extracting metadata and text from thousands of different file formats.
- Content Type Detection - Automates the identification of media types by analyzing byte patterns, filenames, and metadata.
- Document Parsing Engines - Converts unstructured documents from complex formats into structured plain text or XHTML.
- Media Metadata Extraction - Retrieves descriptive attributes and system metadata from images, audio, video, and scientific formats.
- MIME Type Detection - Identifies the format and media type of a file by analyzing byte patterns and file metadata.
- MIME Type Detectors - Includes a specialized detector that identifies document formats using magic bytes and metadata to determine the correct media type.
- Digital Asset Metadata Retrieval - Pulls descriptive attributes and system properties from images, audio, video, and scientific file formats.
- Magic Byte File Identification - Identifies file types by analyzing header magic bytes to determine the correct parser for extraction.
- Optical Character Recognition - Integrates with external OCR tools to extract text from image files and scanned PDFs.
- Named Entity Recognition - Extracts specific named entities such as people and organizations from text using external recognition services.
- Language Identification - Analyzes text within documents to identify the natural language used when metadata is missing.
- Natural Language Processing Analysis - Integrates OCR and language detection to perform linguistic analysis and process text from images.
- Memory-Efficient Chunking - Processes extracted content in small segments via custom handlers to maintain a low memory footprint.
- Sentiment Analysis Tools - Classifies the emotional tone of documents by processing text through integrated natural language analysis tools.
- Custom Extractor Implementations - Provides interfaces for implementing custom logic to extract text and metadata from unsupported file formats.
- Visual Content Analysis - Identifies objects and visual elements within images and videos by integrating with external recognition frameworks.
- Email Text Extraction - Extracts individual messages and attachments from mailbox formats such as mbox, PST, and MSG.
- OCR Integration Gateways - Provides an interface for connecting to external text recognition tools to extract text from image files.
- Multi-Format Archive Extraction Commands - Decompresses various packaging formats like Zip, Tar, and RAR to extract nested document streams.
- Email Archive Processing - Extracts individual messages and embedded attachments from mailbox formats like PST and mbox.
- Parsing Behavior Configurations - Enables control over parser selection and priority via configuration files to override default extraction behaviors.
- Container Format Inspection - Analyzes files wrapped in common container formats to determine the specific document type residing inside.
- Detection Pipeline Configurations - Allows users to define active detectors and their execution sequence to determine the MIME type of a document.
- Structural Event Streaming - Converts documents into a stream of events to preserve structural elements without loading the full file into memory.
- Memory-Efficient Streaming - Processes extracted text in small segments via custom handlers to minimize memory usage for large documents.
- Recursive Document Unpacking - Decompresses nested container formats to extract and parse embedded document streams and attachments.
- Embedded File Extraction - Retrieves and saves embedded files and attachments found within documents to a local directory.
- External Process Delegation - Delegates complex tasks like OCR and translation to external third-party binary tools.
- Recursive Batch Processing - Automates the extraction of content and metadata for all files within a directory using multi-threading.
- Encoding Extraction - Automatically determines the character encoding of plain text documents to ensure accurate extraction.
- Parser Mapping Interfaces - Maps detected MIME types to specialized parser classes through a unified interface for consistent text extraction.
