openvenues/libpostal
Libpostal
Libpostal est une bibliothèque C conçue pour l'analyse et la normalisation d'adresses internationales. Elle utilise le NLP statistique et un classificateur de langue pour décomposer des chaînes d'adresses mondiales non structurées en composants structurés, et pour standardiser les adresses postales en développant les abréviations et en résolvant les variations de nommage régionales dans plusieurs langues.
Le projet fournit des outils de translittération de texte, convertissant divers scripts en formes standardisées Latin-ASCII ou NFD. Il inclut également des capacités de déduplication d'adresses, utilisant une correspondance floue (fuzzy matching) symétrique pour identifier si différents enregistrements d'adresses font référence au même emplacement physique.
La bibliothèque couvre des besoins plus larges de traitement de texte tels que la normalisation UTF-8 et la conversion de nombres écrits en toutes lettres et de chiffres romains en représentations numériques standard. Elle permet des extensions pour la reconnaissance d'adresses via des fichiers de configuration externes afin d'ajouter de nouvelles langues et synonymes.
Features
- Address Parsing - Decomposes unstructured global address strings into structured components like house numbers, street names, and cities.
- Address Language Classifiers - Detects the language of an address string to trigger appropriate region-specific parsing and normalization logic.
- Probabilistic Sequence Taggers - Uses probabilistic NLP models to predict address component boundaries and labels based on open geo data.
- Language Detection - Detects the language of address strings to ensure the correct normalization and parsing rules are applied.
- Address Normalizations - Standardizes unstructured address inputs into a consistent, canonical format for improved search and matching.
- Abbreviation Expansions - Provides multi-language expansion of shorthand address terms into their full-form equivalents.
- Language Classifiers - Provides a statistical language classifier to detect address languages and apply region-specific parsing rules.
- Data Cleaning Procedures - Prepares raw location text for databases by removing noise and standardizing regional naming conventions.
- Physical Address Deduplication - Compares address records using exact or fuzzy matching to determine if they refer to the same physical location.
- Fuzzy Matching - Employs distance metrics to identify duplicate addresses despite spelling or formatting variations.
- Script Transliterations - Converts non-Latin scripts into standardized ASCII representations using predefined transformation maps.
- Geographic Parsing Rules - Applies distinct normalization and parsing logic based on the detected language and geographic region of the address.
- Unicode Normalizers - Standardizes text by decomposing characters to NFD form and stripping accent marks.
- Fuzzy Entity Deduplicators - Identifies and merges address records that refer to the same real-world physical location using fuzzy matching.
- C Geospatial Libraries - Statistical NLP library for parsing and normalizing international addresses.
- Low Level Geospatial Libraries - Parses and normalizes international street addresses.
- Windows Environments - Listed in the “Windows Environments” section of the Awesome C awesome list.
