Awesome GitHub RepositoriesData Normalization

This project provides a structured dataset of the administrative divisions of China, covering five levels from provinces down to villages. It delivers this geographical information in a standardized JSON format designed for data exchange and integration. The dataset is organized as a hierarchical source specifically for building cascading address selectors and region pickers. It uses linked data to enable sequential filtering from higher-level provinces down to village-level boundaries. The project covers geographic data management and regional data normalization. It provides the necessary m

Valibot is a modular, type-safe schema library for validating and parsing structural data in TypeScript environments.

Libpostal is a C library designed for international address parsing and normalization. It utilizes statistical NLP and a language classifier to decompose unstructured global address strings into structured components and standardize street addresses by expanding abbreviations and resolving regional naming variations across multiple languages. The project provides tools for text transliteration, converting various scripts into standardized Latin-ASCII or NFD forms. It also includes capabilities for address deduplication, using symmetric fuzzy matching to identify whether different address reco

qsv is a high-performance command line toolkit for querying, transforming, and analyzing comma-separated value files. It functions as a data wrangling interface and a tabular data profiler, featuring a query engine capable of executing SQL statements and joins directly on flat files without requiring a database. The project is distinguished by its ability to process massive datasets that exceed available system memory. This is achieved through disk-based external memory processing, including multithreaded merge sorting, on-disk hash tables for deduplication, and lightweight file indexing for