openvenues/libpostal
Libpostal
Libpostal is a C library designed for international address parsing and normalization. It utilizes statistical NLP and a language classifier to decompose unstructured global address strings into structured components and standardize street addresses by expanding abbreviations and resolving regional naming variations across multiple languages.
The project provides tools for text transliteration, converting various scripts into standardized Latin-ASCII or NFD forms. It also includes capabilities for address deduplication, using symmetric fuzzy matching to identify whether different address records refer to the same physical location.
The library covers broader text processing needs such as UTF-8 normalization and the conversion of written-out numbers and Roman numerals into standard numeric representations. It allows for extensions to address recognition through external configuration files to add new languages and synonyms.
Features
- Address Parsing - Decomposes unstructured global address strings into structured components like house numbers, street names, and cities.
- Address Language Classifiers - Detects the language of an address string to trigger appropriate region-specific parsing and normalization logic.
- Probabilistic Sequence Taggers - Uses probabilistic NLP models to predict address component boundaries and labels based on open geo data.
- Language Detection - Detects the language of address strings to ensure the correct normalization and parsing rules are applied.
- Address Normalizations - Standardizes unstructured address inputs into a consistent, canonical format for improved search and matching.
- Abbreviation Expansions - Provides multi-language expansion of shorthand address terms into their full-form equivalents.
- Language Classifiers - Provides a statistical language classifier to detect address languages and apply region-specific parsing rules.
- Data Cleaning Procedures - Prepares raw location text for databases by removing noise and standardizing regional naming conventions.
- Physical Address Deduplication - Compares address records using exact or fuzzy matching to determine if they refer to the same physical location.
- Fuzzy Matching - Employs distance metrics to identify duplicate addresses despite spelling or formatting variations.
- Script Transliterations - Converts non-Latin scripts into standardized ASCII representations using predefined transformation maps.
- Geographic Parsing Rules - Applies distinct normalization and parsing logic based on the detected language and geographic region of the address.
- Unicode Normalizers - Standardizes text by decomposing characters to NFD form and stripping accent marks.
- Fuzzy Entity Deduplicators - Identifies and merges address records that refer to the same real-world physical location using fuzzy matching.
- C Geospatial Libraries - Statistical NLP library for parsing and normalizing international addresses.
- Low Level Geospatial Libraries - Parses and normalizes international street addresses.
- Windows Environments - Listed in the “Windows Environments” section of the Awesome C awesome list.
