# quickwit-oss/tantivy

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/quickwit-oss-tantivy).**

15,442 stars · 931 forks · Rust · MIT

## Links

- GitHub: https://github.com/quickwit-oss/tantivy
- awesome-repositories: https://awesome-repositories.com/repository/quickwit-oss-tantivy.md

## Topics

`rust` `search-engine`

## Description

Tantivy is a library for building full-text search engines and indexing frameworks. It provides the core components necessary to organize large collections of text data into searchable structures, enabling the execution of complex queries and the retrieval of information across structured document sets.

The engine utilizes an inverted index architecture to map terms to document identifiers, supported by a segment-based storage model that balances search performance with write throughput. It incorporates specialized data structures, including finite state transducers for term dictionaries and columnar storage for field values, to facilitate efficient retrieval, sorting, and analytical aggregation.

Beyond basic keyword matching, the library supports analytical tasks such as grouping and counting results to reveal data patterns. It leverages processor-level vector instructions and memory-mapped file access to optimize the execution of boolean queries and disk input-output operations.

## Tags

### Data & Databases

- [Full-Text Search Engines](https://awesome-repositories.com/f/data-databases/full-text-search-engines.md) — A high-performance library for building search engines that index structured documents and execute complex full-text queries. ([source](https://github.com/quickwit-oss/tantivy/tree/main/doc/src/))
- [Inverted Index Engines](https://awesome-repositories.com/f/data-databases/data-engineering-infrastructure/data-persistence-storage/specialized-storage-engines/inverted-index-engines.md) — A core indexing framework that organizes large collections of text data into searchable structures for efficient information retrieval.
- [Document Indexing Engines](https://awesome-repositories.com/f/data-databases/data-processing-pipelines/data-processing/search-engines/document-indexing-engines.md) — Organizes text data into a searchable format based on a predefined schema to enable fast retrieval across large document collections. ([source](https://github.com/quickwit-oss/tantivy/tree/main/doc/src/))
- [Structured Data Schemas](https://awesome-repositories.com/f/data-databases/structured-data-schemas.md) — Organizes complex datasets into searchable schemas to enable efficient retrieval and analysis across massive document stores.
- [Finite State Transducers](https://awesome-repositories.com/f/data-databases/search-indexing-technologies/search-indexing/indexing-architectures/finite-state-transducers.md) — Uses compressed graph structures to minimize memory usage while enabling fast prefix and range lookups for term dictionaries.
- [Search and Analytics Engines](https://awesome-repositories.com/f/data-databases/search-indexing-technologies/search-indexing/search-information-retrieval/search-engine-platforms/search-and-analytics-engines.md) — Provides tools for aggregating and sorting search results to enable data insights and structured document analysis.
- [Segmented Storage Architectures](https://awesome-repositories.com/f/data-databases/segmented-storage-architectures.md) — Organizes the index into immutable, independent segments that are merged periodically to balance search performance and write throughput.
- [Columnar Storage Engines](https://awesome-repositories.com/f/data-databases/columnar-storage-engines.md) — Stores document field values in contiguous blocks to facilitate efficient retrieval for sorting, faceting, and analytical aggregation.
- [Result Ordering and Limiting](https://awesome-repositories.com/f/data-databases/result-ordering-and-limiting.md) — Orders query results by relevance or specific field values to ensure users see the most significant information first.
- [Search Result Aggregators](https://awesome-repositories.com/f/data-databases/search-result-aggregators.md) — Groups and counts search results by specific fields to reveal data patterns and provide analytical insights. ([source](https://github.com/quickwit-oss/tantivy/tree/main/doc/src/))