Projects sharing features with Snips Nlu

Knwl.js is a JavaScript named entity recognition library and rule-based text parser. It serves as an extensible information extraction tool designed to identify and pull structured entities, such as dates, times, and locations, from unstructured text strings. The library allows for the definition of specialized rules and custom plugins to identify and extract unique pieces of information. This extensibility enables the automation of information retrieval by converting human-readable text into structured formats for applications and databases. The system utilizes regular expression matching a

ESPnet is a comprehensive speech processing toolkit and PyTorch-based trainer designed for building end-to-end speech recognition, synthesis, and translation models. It provides a structured framework for developing automatic speech recognition systems using transducer and encoder-decoder architectures, alongside engines for text-to-speech synthesis and speech translation pipelines. The project distinguishes itself through a recipe-based workflow execution system that ensures experimental reproducibility by running standardized sequences of scripts for data preparation and model training. It

Ai-chatbot-framework is a conversational AI platform designed for building, training, and managing virtual assistants. It features a natural language processing engine that recognizes user intents and extracts named entities using machine learning models and word embeddings to drive dialogues. The platform supports end-to-end agent development lifecycles alongside visual scenario authoring interfaces for creating and configuring conversation flows. The platform includes stateful multi-turn session stores that preserve contextual history across exchanges, as well as a modular tool execution en

Named entity recognition is a natural language processing library that implements statistical and neural sequence labeling models to extract entities from text. The toolkit provides implementations for hidden Markov models, conditional random fields, and bidirectional recurrent neural networks combined with conditional random field layers. The library supports training machine learning models on annotated training corpora using maximum likelihood estimation for parameter and transition structure estimation. It includes ensemble majority voting consensus strategies to combine independent outpu

Duckling is a deterministic named entity recognizer and natural language entity extractor. It transforms unstructured text into machine-readable data by mapping language-specific input strings to universal structured formats. The system utilizes a rule-based engine and composable language rules to resolve entities without relying on probabilistic models. It supports multilingual text parsing across various regional locales, employing a logic-driven approach to normalize diverse natural language expressions into standardized numeric values. The project covers the extraction and normalization

This is a Chinese natural language processing toolkit providing a suite of tools for word segmentation, part-of-speech tagging, and named entity recognition. It includes a neural dependency parser for analyzing syntactic and semantic relationships between words and a machine learning training suite for creating custom linguistic models using annotated datasets. The toolkit distinguishes itself through its deployment flexibility, offering a dockerized server and a web service interface that exposes processing capabilities via API. It supports the use of pretrained models and allows for the int

DeepPavlov is a conversational AI framework and deep learning NLP library designed for building end-to-end dialogue systems and chatbots. It functions as an NLP pipeline orchestrator that allows users to compose pre-trained models and text processing components into sequential data flows for complex linguistic tasks. The system is distinguished by its ability to act as a chatbot deployment server, exposing trained conversational models as web services via REST and Socket APIs. It utilizes JSON-based pipeline configurations and dynamic variable interpolation to decouple model logic from infras

DeepKE is a knowledge extraction toolkit and framework designed to transform unstructured text into structured knowledge graphs. It provides a pipeline for identifying and classifying named entities, semantic relations, and events, converting raw datasets into structured triples. The project utilizes large language models as tool callers through a standardized context protocol to drive automated data extraction processes. It supports schema-driven extraction across multiple domains and bilingual text, employing joint entity and relation extraction to identify components in a single structured

JioNLP is a Chinese natural language processing toolkit designed for cleaning, normalizing, and extracting structured information from unstructured text. It functions as a linguistic analyzer for Chinese characters and a rule-based named entity extractor, providing a specialized system for sentiment scoring and synthetic data generation for machine learning workflows. The project features a lexicon-based sentiment analysis engine that computes numerical emotional tone scores and a data augmentation library that uses back-translation and synonym replacement to expand training datasets. It incl

PyText is an extensible PyTorch-based framework for building, training, and deploying custom natural language processing models, including text classifiers, sequence taggers, and intent-slot predictors. It provides a modular toolkit that allows developers to assemble these models using pluggable registries for model architectures, data formats, and tensorizers, all configurable through YAML files without requiring code changes. The framework distinguishes itself through its comprehensive support for the full NLP model lifecycle, from training to production inference. It includes pre-built neu

nlp-recipes is a collection of implementation guides and reference templates for applying natural language processing techniques to real-world tasks. It provides standardized workflows and code examples for developing NLP pipelines, from dataset preparation and model training to performance evaluation. The project focuses on the practical application of transformer-based models, offering patterns for fine-tuning pretrained architectures for tasks such as text classification, named entity recognition, and question answering. It also includes a toolkit for model interpretability, allowing users

This project is a curated collection of Chinese names, surnames, and kinship terms designed for linguistic analysis and natural language processing. It functions as a multilingual name dataset and a training resource for named entity recognition, providing a unified repository of names across Chinese, Japanese, and English languages. The project includes a synthetic name generator that creates realistic person names by applying analyzed naming patterns and demographic data. It also provides a cleaned Chinese idiom lexicon gathered and deduplicated from multiple sources. The available data su

Rasa is a chatbot development platform and conversational AI framework used to design, deploy, and integrate multi-turn conversational agents. It functions as an LLM orchestration engine and NLU dialogue manager, combining large language model fluency with structured business logic to control agent behavior. The framework enables the development of conversational assistants that automate text and voice interactions. It allows for the definition of conversational flows using flexible sequences and provides tools to inspect agent decisions to debug and validate the internal reasoning process.

Compromise is a natural language processing library and rule-based text parser designed to analyze unstructured text. It functions as a toolkit for identifying parts of speech, linguistic patterns, and semantic meaning, while providing specialized engines for named entity recognition and the parsing of temporal and numeric data. The project is distinguished by its linguistic morphological engine, which can conjugate verbs across different tenses and inflect nouns and adjectives. It further allows for linguistic model customization through a plugin system that enables the extension of lexicons

Natural is a natural language processing library for Node.js that provides tools for text analysis, tokenization, and phonetic matching. It functions as a collection of specialized toolsets for word stemming, string similarity quantification, and pattern-based text classification. The library includes a phonetic sound analyzer that converts words into phonetic representations to identify matches based on sound rather than literal spelling. It also features a text classification engine that assigns categories to text inputs using trained models and pattern recognition. Additional capabilities

CoreNLP is a Java natural language processing library designed to convert raw human language text into structured data. It utilizes a suite of linguistic annotators to analyze text through a pipeline, extracting grammatical structures, sentiment, and linguistic patterns. The project includes a coreference resolution engine that links multiple mentions of the same entity to maintain contextual consistency across documents. It also provides tools for named entity recognition to categorize people, companies, and locations, and a part-of-speech tagger to assign grammatical categories and base for

QASystemOnMedicalKG is a medical knowledge graph question answering system designed to retrieve disease-centered information from a structured data store. It functions as both a constructor for building medical knowledge graphs and a retrieval system that extracts answers regarding symptoms, causes, and treatments. The system employs a pipeline that converts unstructured medical web data into a graph database using dictionary-based entity segmentation. It utilizes query-based intent classification to parse natural language inputs and maps these queries to specific nodes and edges within the g

Chinese-BERT-wwm is a pre-trained transformer model and encoder designed for Chinese natural language processing. It converts Chinese text into dense vector representations to be used across various natural language processing applications. The model utilizes a whole word masking strategy during pre-training, masking entire words rather than individual characters. This approach is designed to improve the capture of semantic meaning and language structure within Chinese datasets. The project covers a range of downstream tasks including text classification, sequence labeling, and reading compr

This project is a high-performance library for converting raw text into tokens and IDs for machine learning models. It functions as a fast text encoder and a text preprocessing pipeline designed to transform strings into numerical representations with high throughput for research and production. The library includes a subword tokenizer trainer used to analyze text datasets and create custom vocabularies using algorithms such as byte-pair encoding and wordpiece. It provides capabilities for subword vocabulary training and text alignment, allowing character offsets to be tracked during normaliz

spaCy is a Python natural language processing framework designed for industrial-scale text processing. It converts raw text into structured data for machine learning pipelines through a combination of statistical language model trainers, transformer-based text processors, and syntactic dependency parsers. The project enables the integration of pretrained transformer architectures to perform complex linguistic analysis and multi-task learning. It also provides a specialized system for neural named entity recognition to identify and categorize key entities within text. The framework covers a b

ChatterBot is a conversational AI framework and machine learning dialogue system used to build bots that generate automated responses. It functions as a multilingual natural language processing library and a vector-based knowledge base, utilizing logic adapters and statistical pattern matching to select the most confident response to user input. The system supports multilingual chatbot training and processing by using a design independent of specific linguistic rules. It employs semantic vector search to retrieve contextually accurate responses from a database of stored conversations and can

Haystack is an orchestration framework designed for building complex search and generative AI pipelines. It functions as an agentic workflow engine, enabling the construction of automated sequences that allow AI agents to perform multi-step reasoning and data analysis. The framework utilizes a modular, component-based architecture that connects processing steps into directed acyclic graphs. By employing a provider-agnostic integration layer, it decouples core logic from specific external AI services and vector databases, allowing for the flexible exchange of underlying technologies. This desi

DeepPavlov is a deep learning conversational AI framework designed for building end-to-end dialog systems and chatbots. It functions as an NLP model training library and a pipeline system that connects multiple natural language processing models into a single operational chain. The framework provides a REST API model server to expose trained deep learning models as web endpoints. This allows conversational agents to be deployed as web services that handle incoming HTTP requests and return predictions. The system covers the full lifecycle of conversational AI development, including NLP pipeli

Basic Utilities for PyTorch Natural Language Processing (NLP)

AllenNLP is a PyTorch-based research library and deep learning language toolkit designed for developing and training neural network architectures for linguistic tasks. It provides a distributed training system that coordinates data and gradients across multiple GPUs and a framework for integrating pretrained transformer architectures. The system distinguishes itself with a dedicated algorithmic bias mitigation tool used to identify and reduce bias in linguistic model predictions. It also includes model influence analysis to interpret predictions by calculating the influence of specific traini

Fast topic modeling platform