Projects sharing features with Analysis Ik

FlashText is a dictionary-based text processor and keyword extraction library designed for high-performance string searching and programmatic content replacement. It provides specialized utilities to identify and retrieve predefined terms from text or swap specific terms with replacement strings across large volumes of data. The project utilizes an Aho-Corasick string search implementation to enable multiple-pattern matching. This approach allows for single-pass text scanning and linear-time complexity search, ensuring processing time depends on the length of the input text rather than the si

This project provides a comprehensive configuration framework for the Rime input engine, designed to standardize and optimize typing behavior for Simplified Chinese. It serves as a modular system for managing large-scale vocabulary databases, custom input schemes, and personalized text transformation rules across different computing environments. The configuration distinguishes itself through a layered approach to dictionary management and input routing. It utilizes modular patch files to override default settings without altering core source files, allowing for flexible customization of cand

Synonyms is a natural language processing library and semantic similarity engine specifically designed for Chinese text. It functions as a word embedding toolkit and tokenizer that extracts semantic meaning and identifies synonyms by calculating the conceptual closeness between words and sentences. The system provides a toolkit for Chinese word embedding and synonym discovery, allowing for the retrieval of semantically similar words to expand vocabulary. It distinguishes itself through a configuration-driven approach to model loading, which supports the integration of custom word embeddings t

This project is a curated collection of Chinese names, surnames, and kinship terms designed for linguistic analysis and natural language processing. It functions as a multilingual name dataset and a training resource for named entity recognition, providing a unified repository of names across Chinese, Japanese, and English languages. The project includes a synthetic name generator that creates realistic person names by applying analyzed naming patterns and demographic data. It also provides a cleaned Chinese idiom lexicon gathered and deduplicated from multiple sources. The available data su

This is a Chinese text segmentation library that converts Chinese characters into their phonetic pinyin representation. It functions as a polyphone disambiguation tool, resolving ambiguous pronunciations for multi-sound characters using word segmentation and context analysis, and also serves as a pinyin sorting utility for ordering Chinese strings alphabetically. The library distinguishes itself through surname-aware pronunciation switching, applying specialized phonetic rules for Chinese surnames with non-standard pronunciations in name contexts. It supports pluggable word segmentation algor

Wukong is a distributed full-text search engine designed for indexing and retrieving text documents. It functions as a customizable search backend that employs a BM25 relevance ranker to order search results based on term frequency and inverse document frequency. The system includes a specialized Chinese text segmenter to break continuous character strings into meaningful words for accurate indexing and retrieval. To handle large datasets and high request volumes, it utilizes a distributed search index that employs hash-based sharding to split documents across multiple nodes. The engine prov

Synonyms is a Chinese natural language processing tool focused on semantic analysis. It provides capabilities for Chinese word segmentation, part-of-speech tagging, and the retrieval of synonyms based on semantic proximity. The project converts words and sentences into numerical vector representations to calculate similarity scores. This allows for the determination of semantic proximity between different phrases and the identification of chatbot intent through sentence comparison. The system also includes tools for automated keyword extraction and importance ranking to identify significant

pinyin-pro is a Chinese pinyin transcription library and text segmentation tool. It converts Chinese characters into pinyin with support for tones, initials, and finals, while resolving polyphonic characters based on context. The project includes a pinyin pattern matching engine that enables searching Chinese text using full spellings, initials, or hybrid phonetic patterns. It also features a pinyin HTML generator that wraps characters and their transcriptions in markup tags for styled web display. The library provides capabilities for Chinese text segmentation, surname pronunciation priorit

LAC is a Chinese lexical analysis engine and toolkit designed for joint word segmentation, part-of-speech tagging, and named entity recognition. It functions as a high-performance system that identifies word boundaries and grammatical categories using trained machine learning models. The project features a lightweight, compiled native runtime that enables on-device natural language processing and embedding into mobile applications. It includes model compression and conversion to optimize for resource-constrained environments and supports multi-threaded parallel execution to increase throughpu

GPT2-Chinese is a Chinese language model implementation based on the GPT-2 architecture. It provides a causal language model trainer and a natural language generation tool designed for training and generating human-like Chinese text sequences. The system integrates a BERT tokenizer to process Chinese corpora into manageable units for machine learning. It enables the development of predictive text models that can generate specific patterns, such as news or poetry, through prompt-based text completion. The project covers a full workflow including text tokenization, model training using a trans

Riot is a Go-based distributed search engine and indexing server designed for full-text indexing and retrieval. It functions as a retrieval system that sorts documents by relevance using BM25 ranking algorithms, term frequency, and inverse document frequency. The engine provides specialized support for the Chinese language, featuring concurrent text segmentation and phonetic Pinyin mapping to match romanized input with characters. It utilizes a distributed architecture that employs hash-based index sharding to balance data load and throughput across multiple server nodes. The system covers a

SnowNLP is a Python library for Chinese natural language processing. It provides tools for text segmentation, sentiment analysis, document classification, and phonetic transliteration. The library includes capabilities for training and saving custom machine learning models for tokenization and sentiment analysis using raw training datasets. It covers a range of linguistic processing areas, including parts of speech tagging, sentence splitting, and text similarity measurement. The toolkit also provides utilities for extracting key information through text summarization and calculating word im

Elasticsearch is a distributed search engine and document store designed for the high-performance indexing and retrieval of massive volumes of unstructured data. It functions as a centralized analytics platform, providing a schema-flexible architecture that organizes information into searchable indices while maintaining global cluster state through a distributed consensus mechanism. The platform distinguishes itself through its integrated approach to observability, security, and advanced analytics. It combines full-text, vector, and hybrid search capabilities with machine learning-driven insi

OpenSearch is a distributed search and analytics engine designed for indexing, searching, and analyzing massive volumes of structured and unstructured data in real time. It functions as a comprehensive platform that integrates enterprise-grade search capabilities, a vector database for high-dimensional similarity lookups, and a unified observability suite for monitoring logs, metrics, and traces across complex distributed environments. The platform distinguishes itself through its support for agentic workflow automation, allowing users to orchestrate multi-agent tasks and integrate foundation

Pretext is a canvas-based text layout engine designed to calculate precise text dimensions and line breaks for custom rendering. It serves as a rich text measurement tool and a cross-browser typography normalizer, enabling the determination of pixel-perfect widths and heights for mixed inline content without relying on browser CSS. The project distinguishes itself through its ability to handle complex typography and dynamic layouts. It implements language-specific segmentation rules for CJK and Hangul scripts and corrects emoji width variances between DOM and canvas rendering. Additionally, i

This project is a collection of supervised and unsupervised machine learning algorithms implemented from scratch using Python. It serves as an educational resource for studying model training, parameter optimization, and the implementation of core predictive models. The library provides a variety of supervised learning tools, including linear and logistic regression, decision trees, and support vector machines. It also features unsupervised learning capabilities for discovering patterns in unlabeled datasets through clustering algorithms. Broad capability areas include ensemble learning thro

OpenCC is a library and command-line tool for converting text between Simplified Chinese, Traditional Chinese, and Japanese Kanji. It operates at both the individual character and multi-character phrase levels, and applies region-specific vocabulary choices for Mainland China, Taiwan, and Hong Kong during conversion. The conversion engine resolves ambiguous character mappings using semantic and contextual rules, normalizes variant character forms for consistent orthography, and sequences multiple dictionary files into a configurable pipeline. It supports embedding custom conversion rules dire

KnowledgeGraphData is a collection of structured datasets and corpora designed to provide a foundational layer for cognitive intelligence and artificial intelligence systems. It primarily consists of large-scale Chinese knowledge graph datasets, including entity-relation data and NLP training sets used to drive semantic understanding and automated question answering. The project focuses on the construction and export of massive entity-attribute-value graphs, organizing knowledge into portable formats. It provides specialized domain partitioning to tailor information retrieval for professional

This project is a CJK input method framework and configuration set designed for the Rime input engine. It provides a comprehensive system of schemas and dictionary packs to optimize Chinese character entry through pinyin and double-pinyin workflows. The framework is distinguished by its use of Lua-powered extensions that add dynamic utilities, such as inline mathematical calculators, automated timestamps, and text formatting, directly to the input interface. It also features refined word libraries and language models specifically tuned to improve prediction accuracy and first-choice hit rates

Harper is a local English linter and grammar checker designed as an IDE writing assistant. It operates as a language server that provides real-time spelling and grammar analysis for markdown and code comments, processing all linguistic data on the local device to ensure privacy and eliminate cloud dependencies. The tool is specialized for technical documentation, featuring the ability to identify and ignore text within code fences and inline blocks to prevent false positives. It allows for personalized configuration through custom dictionaries and the use of suppression comments to exclude sp

This project is a knowledge base plugin and RAG context manager that uses a local vector database interface to enable semantic search and relationship mapping. It transforms text into numerical vectors to find semantically related notes and excerpts based on conceptual meaning rather than keyword matches. The system differentiates itself through a semantic graph visualizer that maps notes into clusters to reveal conceptual connections. It also features a context manager capable of bundling local notes and excerpts into reusable packs to provide grounded factual bases for large language model

ECDICT is a collection of structured linguistic datasets and an English-Chinese dictionary database. It provides bilingual word definitions, phonetic symbols, and parts of speech, alongside a bilingual geographic gazetteer that maps English place names to Chinese equivalents. These resources are available as a multi-format lexicon export in CSV, SQL, StarDict, and MDX formats. The project distinguishes itself by integrating a linguistic corpus dataset that includes word frequency rankings and academic syllabus markers derived from national corpora. It functions as an educational vocabulary re

Saladict is a browser extension designed to provide instant word definitions and translations by aggregating multiple online dictionary sources. It functions as a multilingual translation tool that retrieves and displays comprehensive word meanings within a unified interface, allowing users to access information without navigating away from their current webpage. The extension features a contextual lookup manager that organizes dictionary sources into profiles. These profiles automatically adjust the active translation sources based on the language or length of the selected text, ensuring tha

Handy is a local speech-to-text automation tool designed to convert spoken audio into text and inject it directly into active desktop applications. By running machine learning models entirely on the host hardware, it provides a private, offline-first environment for dictation and command execution. The system functions as a background service that manages microphone input, transcription state, and text output, enabling hands-free typing across various software environments. The project distinguishes itself through a modular pipeline that integrates local language models for post-transcription

Sonic is a high-performance, lightweight search backend designed to provide real-time full-text search and autocomplete capabilities for applications. It functions as a persistent indexing server that maps text terms to object identifiers, allowing developers to integrate rapid search functionality without storing raw document content directly within the search engine. The system distinguishes itself through a specialized graph-based index that enables real-time word prediction and typo correction. Communication is handled via a custom, low-latency binary protocol over raw TCP sockets, which

Reader is an AI data ingestion pipeline and web content parser designed to convert websites and documents into clean markdown for use with large language models. It functions as a headless browser content extractor and web-to-markdown converter, transforming URLs and PDF files into structured text formats while removing irrelevant web clutter. The system optimizes retrieval augmented generation by acting as a search optimizer that retrieves web results and applies re-ranking to improve context relevance. It further enhances content accessibility by using vision models to generate descriptive

OptiKey is an assistive technology suite and gaze-based input system designed to provide computer access and communication tools. It functions as an eye-tracking virtual keyboard and augmentative and alternative communication software, allowing users to perform text entry and system control via gaze-tracking hardware or other pointer devices. The system differentiates itself through an input-agnostic selection layer that supports diverse sources, including eye trackers, game controllers, and head mice. It utilizes a dwell-time trigger mechanism and gaze fixation selection to translate eye mov