awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
konsheng avatar

konsheng/Sensitive-lexicon

0
View on GitHub↗
3,137 stars·348 forks·mit·24 viewsgithub.com/konsheng/Sensitive-lexicon↗

Sensitive Lexicon

Sensitive-lexicon is a sensitive word detection service and content moderation tool designed to identify prohibited text. It utilizes a curated lexicon of thousands of categorized terms and a fuzzy matching text scanner to detect restricted words and phrases.

The project features specialized filters for Chinese language content across political, social, and adult domains. It supports approximate string matching to identify terms that use noise characters or whitespace to evade standard keyword filters.

The system includes a network interface for hosting the detection service, allowing for real-time lexicon updates without interrupting the active process. It organizes sensitive terms into domain labels to provide context for flagged text.

Features

  • Sensitive Word Filters - Implements a network interface for detecting and filtering banned terms using fuzzy matching and dynamic lexicons.
  • Moderation Tools - Provides a specialized moderation tool for detecting sensitive and prohibited text within Chinese communications.
  • Content Moderation Tools - Identifies and filters inappropriate or restricted text in user-generated content to maintain community safety.
  • Content Moderation Lexicons - Ships a curated database of thousands of categorized sensitive terms to automate identification of inappropriate content.
  • Fuzzy Matching - Uses fuzzy matching algorithms to identify sensitive terms that attempt to bypass filters via noise characters or whitespace.
  • Chinese Text Moderation - Detects prohibited terms and sensitive phrasing within Chinese text across various political and social domains.
  • Content Moderation Filters - Detects inappropriate text across specialized political, social, and adult domains using categorized lists.
  • Chinese Language Content Filters - Offers specialized detection capabilities to identify prohibited Chinese text across multiple domains.
  • Categorized Keyword Mappings - Organizes sensitive terms into distinct domain labels to provide context for flagged text.
  • Lexicon Management - Manages the storage and organization of restricted word lists with support for real-time updates.
  • Finite State Machine Engines - Employs an Aho-Corasick finite state machine engine to search for multiple patterns simultaneously in linear time.
  • Lexicon Storage Tries - Stores sensitive terms in a prefix tree to enable fast lookups and efficient memory usage.
  • Dynamic Configuration Reloading - Supports updating the internal search tree from an external source in real time without restarting the service.

Star history

Star history chart for konsheng/sensitive-lexiconStar history chart for konsheng/sensitive-lexicon

How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI

Open-source alternatives to Sensitive Lexicon

Similar open-source projects, ranked by how many features they share with Sensitive Lexicon.
  • zongzibinbin/mallchatzongzibinbin avatar

    zongzibinbin/MallChat

    4,648View on GitHub↗

    MallChat is an e-commerce backend system and real-time messaging platform. It provides a server-side infrastructure for managing online stores, including integrated shopping carts, order processing, and payment workflows. The system features a WebSocket chat server for instant customer communication and a content moderation engine that uses pattern matching to block sensitive language. It integrates large language models to provide automated conversational AI chatbots for customer support and product recommendations. Identity management is handled through a token-based authentication system

    Java
    View on GitHub↗4,648
  • hit-scir/ltpHIT-SCIR avatar

    HIT-SCIR/ltp

    5,253View on GitHub↗

    This is a Chinese natural language processing toolkit providing a suite of tools for word segmentation, part-of-speech tagging, and named entity recognition. It includes a neural dependency parser for analyzing syntactic and semantic relationships between words and a machine learning training suite for creating custom linguistic models using annotated datasets. The toolkit distinguishes itself through its deployment flexibility, offering a dockerized server and a web service interface that exposes processing capabilities via API. It supports the use of pretrained models and allows for the int

    Pythonchinese-nlpmachine-learningnatural-language-processing
    View on GitHub↗5,253
  • houbb/sensitive-wordhoubb avatar

    houbb/sensitive-word

    5,896View on GitHub↗

    This project is a high-performance Java library and content moderation framework designed to detect and mask prohibited words in text. It utilizes a Deterministic Finite Automaton (DFA) scanner to implement efficient longest-match word detection. The engine distinguishes itself through a text normalization pipeline and noise-filtering preprocessor that standardize character casing, scripts, and widths while removing interspersed special characters to prevent filter evasion. It supports dynamic dictionary management, allowing blacklists and allow-lists to be updated in the background without r

    Javadfadirty-wordfilter
    View on GitHub↗5,896
  • lotabout/skimlotabout avatar

    lotabout/skim

    6,850View on GitHub↗

    Skim is an interactive text filter and terminal selection tool written in Rust. It functions as a command line interface utility that processes input streams to isolate specific entries through real-time user queries and sorting. The tool differentiates itself through ANSI compatibility, allowing it to parse color codes and maintain text formatting during the search process. It supports multiple matching strategies, including configurable fuzzy matching algorithms and regular expression integration. The application covers a broad range of capabilities including field-specific filtering, resu

    Rust
    View on GitHub↗6,850
See all 30 alternatives to Sensitive Lexicon→

Frequently asked questions

What does konsheng/sensitive-lexicon do?

Sensitive-lexicon is a sensitive word detection service and content moderation tool designed to identify prohibited text. It utilizes a curated lexicon of thousands of categorized terms and a fuzzy matching text scanner to detect restricted words and phrases.

What are the main features of konsheng/sensitive-lexicon?

The main features of konsheng/sensitive-lexicon are: Sensitive Word Filters, Moderation Tools, Content Moderation Tools, Content Moderation Lexicons, Fuzzy Matching, Chinese Text Moderation, Content Moderation Filters, Chinese Language Content Filters.

What are some open-source alternatives to konsheng/sensitive-lexicon?

Open-source alternatives to konsheng/sensitive-lexicon include: zongzibinbin/mallchat — MallChat is an e-commerce backend system and real-time messaging platform. It provides a server-side infrastructure… hit-scir/ltp — This is a Chinese natural language processing toolkit providing a suite of tools for word segmentation, part-of-speech… rapidfuzz/rapidfuzz — RapidFuzz is a C++ accelerated Python library providing high-performance string comparison and similarity… houbb/sensitive-word — This project is a high-performance Java library and content moderation framework designed to detect and mask… maxbachmann/rapidfuzz — This is a Python fuzzy string matching library used for calculating string similarity and edit distances. It serves as… lotabout/skim — Skim is an interactive text filter and terminal selection tool written in Rust. It functions as a command line…