google-research-datasets/natural-questionsArchived
Natural Questions
Natural Questions is a large-scale machine learning research dataset designed for training and evaluating open-domain question answering systems. It consists of a corpus of real search queries paired with human-annotated Wikipedia document spans, providing a standardized foundation for advancing automated information retrieval and comprehension technologies.
The project distinguishes itself by providing high-quality ground truth data that supports multiple answer formats, including binary, short-form, and long-form responses. By incorporating extractive span annotations and structured document representations, it enables models to learn how to locate precise information within complex source materials rather than relying on simple text matching.
The dataset includes comprehensive tooling for benchmarking and validation, allowing researchers to measure model performance against verified reference data. It supports the entire lifecycle of question answering development, from training models on annotated queries to executing automated evaluation pipelines that ensure consistent and reproducible results.
