Awesome GitHub RepositoriesNon-Standard UTF-8 Decoding

python-ftfy is a Unicode text repair library designed to fix mojibake and encoding glitches. It provides utilities for byte encoding detection, HTML entity decoding, and the recovery of corrupted text to restore it to its intended Unicode form. The project distinguishes itself through a multi-layered decoding pipeline that identifies and reverts complex encoding mix-ups. It uses heuristic-based detection to resolve instances where text was decoded using the wrong codec across multiple layers of corruption, and it can handle non-standard UTF-8 variants and sloppy encoding mappings. The librar

This project is a Unicode text repair tool and mojibake correction library designed to fix encoding glitches and restore original characters from mangled strings. It functions as a text encoding detector and a Unicode normalization tool to resolve issues where text has been incorrectly decoded. The library specializes in reversing multi-layered encoding errors and repairing complex mojibake patterns. It includes capabilities for detecting lossy encoding sequences, guessing byte encodings, and decoding non-standard UTF-8 variants. The toolset covers a broad range of text cleaning and normaliz