rapidfuzz/RapidFuzz
RapidFuzz
RapidFuzz is a C++ accelerated Python library providing high-performance string comparison and similarity calculations. It functions as a fuzzy string matching toolkit used to quantify the difference between text sequences through Levenshtein distance and other edit distance metrics.
The library focuses on scalable approximate text matching, enabling the identification and ranking of similar strings within large datasets. It provides specialized utilities for finding the best matches in a collection and generating pairwise similarity matrices.
The project covers a broad surface of text processing capabilities, including string similarity scoring, edit distance calculation, and text preprocessing. These tools are used for tasks such as data deduplication, typo correction, and search query suggestion.
Features
- Fuzzy Matching - Provides a comprehensive suite of algorithms for fuzzy string matching to identify close matches despite typos or variations.
- Text Similarity Scoring - Calculates numerical similarity scores between pieces of text using structural and token-based algorithmic comparison.
- C++ Implementations - Ships a core engine written in C++ to provide high-performance string matching and similarity calculations.
- High-Performance C++ Libraries - Provides a high-performance C++ library core exposed to Python for scalable string comparison.
- Edit Distance Calculators - Implements high-performance edit distance calculators to measure the minimum operations required to transform one string into another.
- String Similarity Metrics - Implements a wide range of string similarity metrics, including Levenshtein, Hamming, and Jaro-Winkler distances.
- Approximate Matching Tools - Ships a toolkit for approximate matching to identify the most similar strings from candidate sets using distance metrics.
- Intra-Dataset Deduplication - Enables the identification and merging of nearly identical records to deduplicate datasets.
- Matching and Ranking Logic - Implements matching and ranking logic to order a list of candidates by similarity to a target string.
- Fuzzy Query Suggestions - Supports the ranking of candidates to generate fuzzy search query suggestions based on user input.
- Vector Distance Kernels - Employs SIMD vector distance kernels to process multiple characters simultaneously for accelerated string distance calculations.
- C-Bindings - Provides a high-performance binding layer that translates Python string objects into C-style arrays for low-latency execution.
- Raw Pointer Access - Uses raw pointer access to manage string buffers and score arrays, eliminating Python object overhead in tight loops.
- Bit-Parallel Implementations - Implements high-performance edit distance calculations using bit-parallelism to process state vectors in single CPU cycles.
- Similarity Matrices - Provides utilities to calculate distance or similarity between two collections of strings as a full matrix or direct mapping.
- Approximate String Searching - Performs approximate string searching to find the closest textual matches for a query within a collection.
- Best Match Extraction - Provides utilities to extract the single most similar string from a list of candidate choices.
