Open-source alternatives to Sumy

This repository provides a collection of deep learning models and neural network architectures built for natural language processing tasks. It functions as a library of pre-trained models designed to process, analyze, and generate human language data using the TensorFlow framework. The project utilizes sequence-to-sequence modeling and layered neural architectures to handle variable-length language data. By employing static dataflow graphing and tensor-based representations, the models execute mathematical operations to transform input features into abstract linguistic meanings. Users can loa

HanLP is a natural language processing library and deep learning framework specifically optimized for the Chinese language, while also functioning as a multilingual text processor. It serves as a toolkit for performing linguistic analysis, semantic understanding, and script conversion. The project distinguishes itself through a dedicated focus on Chinese linguistic structures, including a specialized script converter for transforming text between Simplified Chinese, Traditional Chinese, and Pinyin. It further supports domain-specific model training to improve the recognition of professional t

Trafilatura is a Python library and command-line tool for extracting clean, structured text and metadata from web pages. It downloads HTML content, identifies the main body of text, and strips away navigation, ads, and other boilerplate, returning the core article content along with fields like title, author, date, and URL. The tool can also extract user comments and test whether a page contains extractable text, making it a general-purpose web text extraction library. What distinguishes Trafilatura from simpler extractors is its configurable extraction pipeline, which offers high-speed, high

Fairseq is a PyTorch toolkit for sequence-to-sequence modeling, specializing in neural machine translation, automatic speech recognition, and large-scale language model training. It provides a framework for processing and aligning diverse data sources, including text, audio, and video, to support tasks such as speech-to-text conversion and multimodal sequence learning. The project is distinguished by its distributed training capabilities, which utilize parameter sharding, mixed-precision training, and CPU offloading to handle models that exceed single-device memory. It also includes specializ

nlp-recipes is a collection of implementation guides and reference templates for applying natural language processing techniques to real-world tasks. It provides standardized workflows and code examples for developing NLP pipelines, from dataset preparation and model training to performance evaluation. The project focuses on the practical application of transformer-based models, offering patterns for fine-tuning pretrained architectures for tasks such as text classification, named entity recognition, and question answering. It also includes a toolkit for model interpretability, allowing users

Transformers.js is a JavaScript library and web machine learning framework designed to run pretrained transformer models directly in the browser. It serves as a client-side inference engine and a wrapper for the ONNX Runtime, enabling the execution of multimodal AI tasks on user devices without the need for a backend server. The library distinguishes itself by providing a unified toolkit for processing text, image, and audio data locally. This architecture supports privacy-preserving model inference and reduces latency by performing all computations on the client's hardware. Its capabilities

KnowledgeGraphData is a collection of structured datasets and corpora designed to provide a foundational layer for cognitive intelligence and artificial intelligence systems. It primarily consists of large-scale Chinese knowledge graph datasets, including entity-relation data and NLP training sets used to drive semantic understanding and automated question answering. The project focuses on the construction and export of massive entity-attribute-value graphs, organizing knowledge into portable formats. It provides specialized domain partitioning to tailor information retrieval for professional

nlp.js is a JavaScript natural language processing library and development framework used to build natural language understanding engines. It provides a toolkit for creating local machine learning models for intent classification and acts as a multilingual text processor that detects languages and normalizes text across various dialects. The framework distinguishes itself by supporting local execution on both servers and mobile devices, enabling chatbot functionality without an internet connection. It features a specialized system for conversational slot filling to collect mandatory informati

Convert HTML to Markdown-formatted text.

a small library for extracting rich content from urls

This project is a comprehensive collection of educational examples and reference implementations for building vision and language models using PyTorch. It serves as a deep learning tutorial covering the end-to-end process of developing neural networks, from initial architecture definition to final production deployment. The repository provides detailed guides on implementing a wide range of domain-specific models, including convolutional neural networks for object detection and segmentation, as well as transformer and recurrent architectures for natural language processing. It emphasizes gene

NewsBlur is an RSS feed aggregator and social news reader that collects and organizes stories from feeds, newsletters, and websites into a single interface. It functions as a feed synchronization service that maintains read states and subscription data across multiple devices and third-party applications. The platform distinguishes itself with AI-powered content summarization to generate briefings and answer questions about articles, alongside a system for training content classifiers. These classifiers learn user preferences for authors and tags to automatically highlight preferred topics or

Campsite is an asynchronous communication tool and integrated workspace suite designed for team coordination and knowledge sharing. It provides a unified interface that combines long-form posts, documentation, and project tracking to reduce the noise associated with real-time chat. The platform is an LLM-powered collaboration environment that uses large language models to generate concise summaries and recaps of extensive conversation threads and meeting transcripts. It further distinguishes itself as a webhook automation gateway, enabling the synchronization of internal discussions with exte

Librum is a cross-platform digital ebook reader and library manager designed to render multiple book formats. It functions as an integrated reading assistant and a system for organizing digital libraries across multiple devices. The project distinguishes itself by providing a self-hosted backend option, allowing users to maintain full control over their book data and metadata via a private server. It also integrates a language model assistant to provide text summarization, explanations, and translations of specific passages while reading. The application covers comprehensive library manageme

SnowNLP is a Python library for Chinese natural language processing. It provides tools for text segmentation, sentiment analysis, document classification, and phonetic transliteration. The library includes capabilities for training and saving custom machine learning models for tokenization and sentiment analysis using raw training datasets. It covers a range of linguistic processing areas, including parts of speech tagging, sentence splitting, and text similarity measurement. The toolkit also provides utilities for extracting key information through text summarization and calculating word im

Ragas is an evaluation framework designed to measure the performance of retrieval-augmented generation pipelines and autonomous agent workflows. It provides a comprehensive suite of tools for benchmarking system outputs, utilizing language models as automated judges to score performance against defined rubrics and reference data. By standardizing inputs, retrieved contexts, and generated responses into a unified schema, the project enables consistent analysis across complex AI applications. The framework distinguishes itself through its ability to generate synthetic test datasets from existin

WechatSogou is a Python library that provides a programmatic bridge to WeChat content by scraping Sogou's search engine results. It resolves WeChat account profile pages to article lists, filters trending articles by predefined categories, and manages browser-like cookie sessions to maintain authenticated access. The library parses raw HTML to extract article metadata, account details, and keyword suggestions, while dynamically adjusting request frequency to avoid IP blocking. The project offers capabilities for searching WeChat public accounts and articles by keyword, returning profile detai

This project is a systematic framework for English language acquisition that applies structured workflows and cognitive strategies to build linguistic proficiency. It focuses on the construction of a linguistic knowledge base, enabling learners to master vocabulary and grammar through methodical training. The methodology is distinguished by its use of computer science concepts, such as mental-model-based learning and memory buffers, to organize progression. It emphasizes a cognitive-translation bypass to develop target language thinking, reducing mental latency by processing information direc

Pup is a command line tool for extracting and filtering data from HTML documents using CSS selectors. It functions as a parser and selector engine that isolates specific elements based on tags, IDs, classes, and attributes. The project provides utilities for converting selected HTML nodes into plain text, attribute values, or structured JSON objects. It includes a markup formatter that corrects missing tags and applies consistent indentation to improve the readability of HTML documents. The tool handles the retrieval of text content and attributes through a CSS selector engine, supporting co

chatGPTBox is a browser extension that integrates large language model chat interfaces and AI tools directly into the web browsing experience. It functions as an AI productivity toolkit and API client, allowing users to access AI assistants via a floating chat interface without leaving their active webpage. The project distinguishes itself by offering context-aware assistance and website-specific adaptations based on the current URL. It further enhances the browsing experience by displaying AI-generated responses alongside standard search engine results and providing a system to route chat re

OpenPDF is a Java library and document processor used for creating, editing, rendering, and encrypting PDF documents. It functions as a toolkit for generating new files from scratch, modifying existing document structures, and extracting text content. The project includes a dedicated engine for transforming HTML and CSS content into PDF documents by parsing markup and applying styles. It also provides a rendering engine to convert PDF pages into image formats for thumbnails and previews, alongside a security utility for protecting content via document encryption. The library supports the add

WriteGPT is an end-to-end essay automation system that combines visual recognition and automated text generation to convert images into finished digital documents. It functions as a creative text generator and document processor, utilizing language models to produce long-form written content and essays. The system integrates a neural text fluency evaluator to score the linguistic quality and naturalness of generated prose. It also includes a transformer-based text summarizer to condense long documents into concise summaries. The project provides a pipeline for optical character recognition t

BestBlogs is an AI-powered content aggregation and curation platform that integrates feed management with automated text analysis. It functions as a centralized reading space that collects updates from diverse digital sources, including RSS feeds and social media, to deliver personalized daily briefings and thematic digests. The platform distinguishes itself by using large language models to process, summarize, and score incoming content. It provides deep insights through generative summarization, thematic synthesis, and interactive question-answering, while also offering multilingual transla

Grobid is a machine learning system designed to transform academic and scientific PDF publications into structured XML. It functions as a PDF to XML parser and scholarly metadata extractor, identifying and normalizing titles, authors, affiliations, and bibliographic references from research papers. The system utilizes a deep learning document segmenter to divide raw PDFs into functional regions and employs a bibliographic reference resolver to match citations against external registries for metadata enrichment and DOI resolution. It supports a full machine learning model training pipeline, al

pdf2htmlEX is a PDF to HTML converter that transforms documents into web pages while preserving the original layout, fonts, and formatting. It functions as a layout engine and text extractor, mapping PDF coordinate data to HTML and CSS to maintain visual fidelity. The tool converts PDF content into searchable and selectable native HTML text by embedding original document fonts. It maintains document interactivity by preserving internal links, bookmarks, and outlines, converting them into functional web navigation. The conversion process supports flexible output structures, allowing documents

docetl is an AI-powered document ETL tool and map-reduce orchestrator designed to transform large collections of unstructured documents into structured, queryable tables using language models. It provides a declarative pipeline framework for extracting, cleaning, and transforming data from sources such as PDFs and text files into predefined schemas. The project distinguishes itself through a semantic data integration suite that enables joining datasets and resolving duplicate entities based on embedding-based similarity. It includes an interactive prompt playground for developing and optimizi

This project is a Model Context Protocol server that enables Large Language Models to control Playwright browsers for web automation, scraping, and end-to-end testing. It functions as a programmable interface for executing JavaScript, capturing screenshots, and interacting with web elements across multiple browser engines. The server exposes browser automation capabilities as a set of standardized tools that models can discover and invoke. It supports session-based browser isolation to ensure unique contexts for each client connection and provides a transport layer using either standard input