Open-source alternatives to This Word Does Not Exist

基于草稿的文本生成模型

Chinese Language Generation Evaluation 中文生成任务基准测评

2022-1-5: Our new work (ScriptWriter-CPre) has been accepted by TOIS! - 2021-10-19: We upload the code and data for our new model ScriptWriter-CPre. - 2020-06-11: We find a minor error in the data thus we upload a new one. We provide the code for building data file from text file. You can now…

This is the code used in the paper Deep Graph Convolutional Encoders for Structured Data to Text Generation by Diego Marcheggiani and Laura Perez-Beltrachini.

PaperRobot: Incremental Draft Generation of Scientific Ideas [Sample Output](https://eaglew.github.io/PaperRobot/)

LLM Transparency Tool (LLM-TT), an open-source interactive toolkit for analyzing internal workings of Transformer-based language models. Check out demo at https://huggingface.co/spaces/facebook/llm-transparency-tool-demo

This project is a high-performance BERT embedding service and inference server designed to map text sequences into fixed-length numerical vectors. It functions as a machine learning microservice and distributed model server that decouples request handling from heavy computation. The system utilizes a ZeroMQ messaging infrastructure to provide low-latency communication between distributed clients and the inference server. It incorporates server-side batch processing and GPU workload scaling to maximize hardware utilization and manage high request volumes. The platform supports semantic search

Make any LLM to think deeper like OpenAI o1 and deepseek R1!

This is a fork of CyberZHG/keras_bert which supports Keras BERT on TPU.

Transformers is a comprehensive library for machine learning that provides a unified interface for training, fine-tuning, and deploying transformer-based models. It supports a wide range of tasks, including text classification, language modeling, question answering, and sequence-to-sequence translation, while offering specialized architectures for both text and vision processing. The framework includes tools for managing the entire model lifecycle, from data preprocessing and tokenization to distributed training and inference. The library features extensive support for model optimization and

Multi-Modal Multi-Task Learning

BertViz is a diagnostic toolset for visualizing attention heads and internal model behaviors to interpret how language models process text. It serves as an interpretability tool and debugger for natural language processing models, specifically providing interactive maps of attention mechanisms within Transformer architectures. The project enables the analysis of token relationships through detailed views of specific attention heads and layers. It supports global attention visualization across all layers, encoder-decoder attention mapping, and the inspection of individual neurons within query

A Model for Natural Language Attack on Text Classification and Inference

A minimalistic codebase for finetuning and interacting with natural language generation models from HuggingFace Transformers using PyTorch Lightning.

Generating multiple choice questions from text using Machine Learning.

Generate Hacker News Comments from Titles

LLM Reasoners is a library to enable LLMs to conduct complex reasoning, with advanced reasoning algorithms. It approaches multi-step reasoning as planning and searches for the optimal reasoning chain, which achieves the best balance of exploration vs exploitation with the idea of "World Model"…

This repo contains end-to-end recipes to pretrain and finetune the BERT (Bidirectional Encoder Representations from Transformers) language representation model using Azure Machine Learning service.

A robust Python tool for text-based AI training and generation using OpenAI's GPT-2 and EleutherAI's GPT Neo/GPT-3 architecture.

Easily train your own text-generating neural network of any size and complexity on any text dataset with a few lines of code, or quickly train on a text using a pretrained model.

Notice: This project uses an older version of TensorFlow, and is no longer supported. Please consider using other latest alternatives.

Read more about AIDungeon2 and how it was built here.

An open-source library for contamination detection in NLP datasets and Large Language Models (LLMs).

Question Generation using 🤗transformers - Project Details - Initial experiments - answer aware question generation - answer extraction models - Multitask QA-QG - End-to-End question generation (answer agnostic) - Results - Requirements - Usage - Question Generation - Multitask QA-QG -…

inferring sql queries from plain-text questions about tables