github-linguist/linguist
Linguist
Linguist is a programming language detection library designed to identify the languages used within source code files and software repositories. It functions as a repository metadata classifier, providing the automated analysis necessary to generate language statistics and insights for version control platforms.
The tool employs a strategy-based detection pipeline that combines multiple identification methods to ensure accuracy. It utilizes heuristic-based pattern matching for file extensions and filenames, supplemented by regex-driven content analysis and Bayesian statistical classification to resolve ambiguities. Users can further refine these results through configuration-driven override logic, which allows for the manual mapping of specific paths or patterns to designated languages.
Beyond basic identification, the library supports comprehensive software project analytics by auditing the composition of codebases. It provides the infrastructure to manage repository metadata, enabling developers to correct or customize language detection settings for specific files or directories to meet project requirements.
Features
- Source Code Language Detectors - Identifies the language of source code files by analyzing extensions, filenames, and content patterns.
- Language Identification Utilities - Examines source code files to determine the underlying programming language by analyzing extensions, filenames, and content patterns.
- Code Analysis Tools - Categorizes and audits codebases by automatically detecting the primary programming languages used in a repository.
- Language Classifiers - Determines language statistics for version control platforms by scanning file contents and applying custom override rules.
- Language Identification Services - Automatically detects programming languages used in a repository to provide accurate statistics and syntax highlighting.
- Codebase Composition Analyzers - Analyzes the composition of codebases to understand the distribution of languages and technologies used across software projects.
- Detection Overrides - Allows users to specify or ignore programming languages for particular files or directories to correct automatic identification inaccuracies.
- Version Control and Repository Tools - Processes file contents within version control systems to generate language breakdowns and insights for hosted repositories.
- Configuration Overrides - Provides configuration-driven override logic to manually map specific file paths or patterns to designated programming languages.
- Regex-Driven Parsers - Applies regular expressions to file contents to identify language-specific syntax patterns when metadata is ambiguous.
- File Pattern Matching - Analyzes file extensions and filenames against a predefined database to determine the primary language of a source file.
- Language Detection Classifiers - Uses Bayesian statistical classification to calculate the probability of a language match based on source code token frequencies.
