wooorm/franc
Franc
Franc is a natural language detection library and command line identifier used to determine the written language of a text sample. It functions as a statistical language profiler that identifies and classifies multilingual text by analyzing character distributions.
The tool employs a trigram-based statistical analysis system, comparing the frequency of three-character sequences in an input sample against reference profiles. It ranks potential language matches by calculating the statistical distance between the input and these profiles, allowing it to return a ranked list of probable languages.
The project provides a command line interface for text analysis and supports automated content classification. It utilizes modular language datasets and precomputed n-gram tables to separate detection logic from language-specific profile data.
Features
- Natural Language Code Detectors - Detects the natural language of text input by analyzing its statistical profile.
- Language Detection - Provides a statistical profiling system to identify and rank the most probable natural languages in a text sample.
- Language Detection Tools - Provides a command line interface to identify the natural language of text samples.
- Language Detection in Chunks - Identifies the natural language of text samples using statistical analysis.
- Probability-Based Ranking - Ranks potential language matches by calculating the statistical distance between input samples and reference profiles.
- Trigram Analysis - Identifies languages by analyzing the frequency of three-character sequences in the provided text.
- N-Gram Detection - Employs n-gram detection by comparing character sequence frequencies against precomputed tables.
- NLP Statistical Profiling - Functions as a statistical profiler that ranks potential language matches based on the input text profile.
- Language Identification CLIs - Provides a terminal interface to analyze text streams and determine their written natural language.
- Text Analysis CLIs - Provides a command line tool for scanning and analyzing the language of text files and streams.
- Automated Classification - Enables automated classification of large amounts of text based on detected language.
- Modular Language Datasets - Implements a modular architecture that separates language-specific profile data from the core detection logic.
- Multilingual Text Processing - Prepares text for further processing by first determining the original natural language of the input.
- Text Classification - Classifies large volumes of content into groups based on their detected language profiles.
- Language Datasets - Utilizes language datasets to provide reference profiles for statistical language identification.
- Language - Returns a ranked list of all potential language matches found within a text sample.
- Natural Language Processing - Language detection for text.
Star history
How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
