Projects sharing features with Chinese Chatbot Corpus

SnowNLP is a Python library for Chinese natural language processing. It provides tools for text segmentation, sentiment analysis, document classification, and phonetic transliteration. The library includes capabilities for training and saving custom machine learning models for tokenization and sentiment analysis using raw training datasets. It covers a range of linguistic processing areas, including parts of speech tagging, sentence splitting, and text similarity measurement. The toolkit also provides utilities for extracting key information through text summarization and calculating word im

This project is a CJK input method framework and configuration set designed for the Rime input engine. It provides a comprehensive system of schemas and dictionary packs to optimize Chinese character entry through pinyin and double-pinyin workflows. The framework is distinguished by its use of Lua-powered extensions that add dynamic utilities, such as inline mathematical calculators, automated timestamps, and text formatting, directly to the input interface. It also features refined word libraries and language models specifically tuned to improve prediction accuracy and first-choice hit rates

This project is a comprehensive collection of country names and standardized ISO 3166-1 codes. It serves as a geographic reference repository that maps global nations to their respective international standards for use in data normalization and identification. The dataset is provided in multiple languages to support internationalization and is distributed in various file formats to ensure compatibility across different programming environments. The project covers capabilities for country metadata mapping, multilingual geographic labeling, and the retrieval of alpha-2 and alpha-3 codes for ad

The OpenTelemetry Collector is a vendor-agnostic proxy and observability data pipeline that receives, processes, and exports traces, metrics, and logs. It functions as a telemetry ingestion gateway and multi-backend monitoring agent, translating various data formats into a standardized internal representation for consistent processing. The project distinguishes itself through a plugin-based component model, allowing the integration of custom receivers, processors, and exporters without modifying the core codebase. It utilizes a configurable pipeline system where telemetry flows through a sequ

DevLake is a DevOps data platform and analytics tool designed to orchestrate data pipelines that ingest, transform, and sync metadata from external development tools into a unified database. It functions as a system for collecting and normalizing data from source control, CI/CD pipelines, and issue trackers into a standardized schema to enable consistent software delivery analytics. The platform distinguishes itself by transforming tool-specific data into a common domain model, allowing for the calculation of engineering metrics via SQL. It provides specialized frameworks for measuring DORA m

UserScripts is a collection of JavaScript browser userscripts designed to modify website behavior and add custom functionality to web browsers. It serves as a multi-purpose toolset for web page content automation, web interface enhancement, and specialized web scraping and downloading. The project distinguishes itself through a wide range of specialized utilities, including a browser-based text transformer for character encoding and terminology mapping, and tools for bypassing content censorship. It provides advanced web scraping capabilities such as deciphering obfuscated download links, agg

Aidoku is a manga reader application and digital library manager. It serves as a modular content aggregator that allows users to discover, download, and read manga from various third-party sources and local files. The application utilizes a modular source plugin system to integrate external provider packages, enabling the ingestion of content from multiple third-party sources. It includes a sync engine that communicates with external tracking APIs to maintain consistent reading progress across different platforms. The system covers manga library management, including the ability to search fo

XChange is a cryptocurrency exchange integration library and API wrapper that provides a unified interface for connecting to multiple cryptocurrency exchanges. It serves as a multi-exchange trading interface and a market data streamer, normalizing raw data from various providers into standardized data objects. The project distinguishes itself through an adapter-based normalization system and a reactive WebSocket streaming model designed to receive real-time price and volume updates while minimizing thread usage. It includes a dedicated authentication handler for managing API keys, request non

mmagic is a multimodal training pipeline and framework for generative AI, focusing on visual synthesis and restoration. It provides the infrastructure to build and train models for tasks such as text-to-image and text-to-video generation, 3D-aware content synthesis, and high-fidelity image translation using diffusion models and generative adversarial networks. The project distinguishes itself through specialized capabilities for generative model personalization, including techniques for fine-tuning subjects and styles. It also supports advanced visual manipulations such as latent space interp

Trime is a customizable text input framework and engine based on the Rime input method. It enables the entry of characters across multiple languages using phonetic markers and shape-based patterns. The project functions as a cross-platform input method, providing the necessary logic to build and deploy text input tools for both mobile and desktop devices. It also serves as a Chinese text converter for translating traditional Chinese characters into simplified Chinese to create localized resource files.

This project is a research data sharing framework and provenance protocol designed to ensure computational reproducibility. It provides a standardized set of guidelines for transforming raw source data into tidy formats through documented processing scripts and cleaning workflows. The framework distinguishes itself by emphasizing a strict provenance-based packaging system. It requires the organization of raw data, processing recipes, and code books into a single package, ensuring that original unmodified sources are preserved to allow for independent verification of all transformation steps.

This project provides a suite of interfaces and tools for accessing electricity carbon intensity and production metrics. It includes an API for real-time and historical data, a geographic power data map for visualizing regional carbon intensity and renewable energy percentages, and a system for extracting datasets required for standardized greenhouse gas emissions reporting. The project features an interactive API sandbox that allows users to test requests and inspect data responses without writing code. It also includes mechanisms for institutional email verification to manage access to hist

dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control. It functions as a data transformation engine that enables users to define data structures and business logic through declarative configuration files, which the system then compiles into executable code. By managing complex data dependencies through a directed acyclic graph, it ensures that transformation tasks execute in the correct order while maintaining a manifest-driven state to track lineage and execution history. The project distinguishes itself through an adapter-based d

This project is a pandas data analysis cookbook and Python data science guide. It provides a collection of programmatic recipes and examples for cleaning, manipulating, and analyzing structured data. The project focuses on providing a containerized analysis environment to ensure a consistent workspace and reproducible dependencies when executing data processing scripts. It covers a broad range of data science capabilities, including data ingestion from external sources, raw data cleaning, and exploratory data analysis. These recipes demonstrate how to perform structured data analysis through

This project is a centralized API specification registry and governance framework. It serves as a source of truth for structured API schemas, gRPC definitions, and data models used to maintain consistency across a large ecosystem of services. The system functions as a multi-language client generator, transforming standardized interface definitions into executable source code and language-specific libraries. This process ensures that client libraries and technical documentation remain synchronized by deriving them directly from the original interface definitions. The project also covers the m

This project is a web-based manga and novel downloader and multi-site web scraper designed to extract images and text from diverse media platforms. It functions as a digital media archiver and EPUB e-book generator, using a plugin-based crawler architecture with site-specific scripts to define how content is extracted from various international websites. The system distinguishes itself through authenticated web crawling, using browser cookie simulation to access restricted or member-only content. It includes specialized capabilities for digital comic archiving, which organizes image sequences

MNBVC is a dataset pipeline and toolkit designed for the collection, cleaning, and normalization of massive text and code corpora used to train large language models. It provides specialized tools for harvesting source code, commit histories, and repository metadata from version control platforms, alongside a multilingual text corpus collector for gathering parallel text and academic papers. The project distinguishes itself through comprehensive capabilities for processing diverse document types, including a PDF-to-text converter that transforms complex layouts and formulas into structured JS

Fcitx5 Android is an input method manager that brings the Fcitx5 framework to Android, enabling multilingual text input through a customizable virtual keyboard. It functions as a platform for loading plugin-based input engines, supports conversion between simplified and traditional Chinese characters, and provides a theme engine for dynamically altering the keyboard's appearance with custom colors, images, and popup previews. The project is built around a native C++ core that handles dictionary lookups and text processing, connected to the Android interface via a JNI bridge for performance. I

Open-Instruct is a distributed training and instruction tuning framework for large language models. It functions as a coordinator for supervised fine-tuning, reinforcement learning from human feedback pipelines, and tool-use training, providing specialized roles for dataset curation and model alignment. The project distinguishes itself through a high-performance training architecture that utilizes actor-based distributed coordination and hybrid sharding to manage large GPU clusters. It implements advanced alignment techniques including direct preference optimization, group relative policy opt

This project is a comprehensive pandas data analysis tutorial and instructional guide designed for learning data manipulation and analysis. It serves as a tabular data processing guide and a manual for time series analysis, providing a structured approach to cleaning, merging, and transforming datasets. The repository functions as a data feature engineering course, providing tutorials on constructing and selecting dataset features to improve machine learning model performance. It also includes a vectorized data operations guide for performing element-wise mathematical computations and matrix

qsv is a high-performance command line toolkit for querying, transforming, and analyzing comma-separated value files. It functions as a data wrangling interface and a tabular data profiler, featuring a query engine capable of executing SQL statements and joins directly on flat files without requiring a database. The project is distinguished by its ability to process massive datasets that exceed available system memory. This is achieved through disk-based external memory processing, including multithreaded merge sorting, on-disk hash tables for deduplication, and lightweight file indexing for

DbGate is a universal database management tool and SQL client that provides a unified interface for querying and administering multiple SQL and NoSQL databases. It functions as a multi-database administration GUI and SQL IDE, allowing users to write and execute scripts and manage database schemas. The project distinguishes itself by acting as an API client and explorer for REST, GraphQL, and OData services, enabling users to fetch and export data from these endpoints. It also serves as a data integration tool, facilitating the movement of records between diverse databases and file formats suc

Cleanlab is a data-centric AI library and toolkit designed to improve machine learning model performance by detecting label errors and increasing overall dataset quality. It implements a confident learning framework that iteratively refines label noise estimates by comparing model predictions with estimated label probabilities to identify mislabeled examples. The project provides specialized utilities for active learning optimization, allowing for the selection of the most impactful examples for labeling or re-labeling. It also includes an outlier detection tool to identify atypical data poin

This project is a development course and learning curriculum focused on building large language model chatbots. It provides a structured series of tutorials for creating conversational agents through the application of natural language processing and deep learning models. The materials include a technical walkthrough for implementing neural networks and word embeddings to handle automated question-answering tasks. It also provides a guide for constructing large-scale conversation corpora from external text sources to train and evaluate dialogue systems. The curriculum covers core text analys

Feast is a machine learning feature store and MLOps data infrastructure layer. It provides a centralized system for managing and serving features across offline training and online production environments, utilizing an online feature serving layer for low-latency retrieval. The project centers on a feature registry that acts as a central catalog for defining, governing, and discovering feature services. It employs a unified data access layer to decouple feature retrieval from physical storage and includes a point-in-time data generator to create historically accurate training datasets that pr

Webmagic is a Java web crawling framework designed for building scalable automated crawlers to download and process large volumes of web pages. It functions as a distributed web crawler and dynamic content crawler, utilizing an XPath HTML parser to locate and extract specific data points from page structures. The framework distinguishes itself through its ability to handle dynamic content by rendering JavaScript and executing asynchronous requests to extract data from non-static pages. It also allows users to define and execute crawler logic via scripting languages, enabling the update of col

This project is a structured learning curriculum and technical reference for mastering deep learning with TensorFlow. It provides a comprehensive guide for building, training, and deploying neural networks, combining theoretical fundamentals with practical implementation examples. The repository distinguishes itself by covering the end-to-end machine learning workflow, from low-level tensor mathematics and linear algebra to the creation of complex model architectures. It includes specific guidance on developing data pipelines for diverse data types, such as images, text, and time-series seque