4 Repos
Awesome GitHub RepositoriesData Normalization
Standardizes raw input formats into a consistent internal representation.
Distinct from Input Validation: Distinct from Input Validation: focuses on the process of cleaning and standardizing data rather than simply verifying correctness.
Explore 4 awesome GitHub repositories matching part of an awesome list · Data Normalization. Refine with filters or upvote what's useful.
Awesome Data Normalization GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- modood/administrative-divisions-of-china
modood/Administrative-divisions-of-China
20,829Auf GitHub ansehenThis project provides a structured dataset of the administrative divisions of China, covering five levels from provinces down to villages. It delivers this geographical information in a standardized JSON format designed for data exchange and integration. The dataset is organized as a hierarchical source specifically for building cascading address selectors and region pickers. It uses linked data to enable sequential filtering from higher-level provinces down to village-level boundaries. The project covers geographic data management and regional data normalization. It provides the necessary m
Standardizes regional inputs using a consistent hierarchy of provinces, cities, districts, towns, and villages.
JavaScriptaddressadministrative-divisionsarea - open-circle/valibot
open-circle/valibot
8,769Auf GitHub ansehenValibot is a modular, type-safe schema library for validating and parsing structural data in TypeScript environments.
Transforms raw input into a standardized format during the validation process.
TypeScriptbundle-sizemodularparsing - openvenues/libpostal
openvenues/libpostal
4,819Auf GitHub ansehenLibpostal ist eine C-Bibliothek für das Parsen und Normalisieren internationaler Adressen. Sie nutzt statistisches NLP und einen Sprachklassifikator, um unstrukturierte globale Adress-Strings in strukturierte Komponenten zu zerlegen und Straßenadressen durch das Auflösen von Abkürzungen und regionalen Namensvariationen über mehrere Sprachen hinweg zu standardisieren. Das Projekt bietet Tools für die Texttransliteration, um verschiedene Schriftsysteme in standardisierte Latin-ASCII- oder NFD-Formen zu konvertieren. Es enthält zudem Funktionen zur Adress-Deduplizierung, wobei symmetrisches Fuzzy-Matching verwendet wird, um zu identifizieren, ob verschiedene Adressdatensätze denselben physischen Ort referenzieren. Die Bibliothek deckt breitere Textverarbeitungsanforderungen ab, wie UTF-8-Normalisierung sowie die Konvertierung von ausgeschriebenen Zahlen und römischen Ziffern in numerische Darstellungen. Sie ermöglicht Erweiterungen der Adresserkennung durch externe Konfigurationsdateien, um neue Sprachen und Synonyme hinzuzufügen.
Provides multi-language expansion of shorthand address terms into their full-form equivalents.
C - dathere/qsv
dathere/qsv
3,687Auf GitHub ansehenqsv is a high-performance command line toolkit for querying, transforming, and analyzing comma-separated value files. It functions as a data wrangling interface and a tabular data profiler, featuring a query engine capable of executing SQL statements and joins directly on flat files without requiring a database. The project is distinguished by its ability to process massive datasets that exceed available system memory. This is achieved through disk-based external memory processing, including multithreaded merge sorting, on-disk hash tables for deduplication, and lightweight file indexing for
Reads data with specific quoting or transcoding rules to standardize input for processing.
Rustaickancsv
