How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.
This is a list of datasets/corpora for NLP tasks, in reverse chronological order. Suggestions and pull requests are welcome. The goal is to make this a collaborative effort to maintain an updated list of quality datasets.
The main features of karthikncode/nlp-datasets are: Datasets and Corpora, Question Answering Datasets.
Open-source alternatives to karthikncode/nlp-datasets include: deepmind/rc-data — Question answering dataset featured in "Teaching Machines to Read and Comprehend. google-research-datasets/natural-questions — Natural Questions is a large-scale machine learning research dataset designed for training and evaluating open-domain… brightmart/nlp_chinese_corpus — This is a large-scale collection of curated Chinese text corpora designed for training natural language processing… chinawithfrank/chatbotcourse — This project is a development course and learning curriculum focused on building large language model chatbots. It… deepmind/narrativeqa — This repository contains the NarrativeQA dataset. It includes the list of documents with Wikipedia summaries, links to… deepmind/aqua.
Question answering dataset featured in "Teaching Machines to Read and Comprehend
Natural Questions is a large-scale machine learning research dataset designed for training and evaluating open-domain question answering systems. It consists of a corpus of real search queries paired with human-annotated Wikipedia document spans, providing a standardized foundation for advancing automated information retrieval and comprehension technologies. The project distinguishes itself by providing high-quality ground truth data that supports multiple answer formats, including binary, short-form, and long-form responses. By incorporating extractive span annotations and structured documen
This is a large-scale collection of curated Chinese text corpora designed for training natural language processing models. The project provides a variety of datasets, including a deduplicated archive of millions of news articles with titles and keywords, high-quality categorized question-and-answer pairs, and parallel translation corpora. The collection includes millions of aligned Chinese and English sentence pairs used for cross-lingual model training and machine translation development. It also contains filtered question-and-answer data organized by label for the construction of knowledge-
This project is a development course and learning curriculum focused on building large language model chatbots. It provides a structured series of tutorials for creating conversational agents through the application of natural language processing and deep learning models. The materials include a technical walkthrough for implementing neural networks and word embeddings to handle automated question-answering tasks. It also provides a guide for constructing large-scale conversation corpora from external text sources to train and evaluate dialogue systems. The curriculum covers core text analys