Projects sharing features with Document Qa

This project is a collection of scripts and workflows for training, fine-tuning, and deploying large language models using the Hugging Face Transformers toolkit. It functions as a distributed training framework, a library for natural language processing task implementations, and a system for building retrieval-augmented generation chatbots. The repository includes specialized tools for model optimization, such as a Bayesian hyperparameter optimizer for automatically tuning model settings. It provides implementations for scaling model training across multiple graphics processors using data par

Reor is a local AI knowledge management application that stores, links, and searches personal notes using large language models and vector embeddings entirely on the user's device. It functions as a private AI note assistant, keeping all data and processing local for full privacy without relying on external cloud services. The application integrates with Ollama to manage the lifecycle of local LLMs and embedding models, handling downloads, updates, and execution. Notes are imported from markdown files, preserving existing file structure, and are automatically linked through vector-similarity

This project provides a framework for executing large language models and performing document-based question answering entirely on local consumer hardware. By integrating a CPU-based inference engine with a local vector database, it enables users to process information without relying on cloud-based APIs or specialized graphics processing units. The system functions as a command-line tool that manages the full lifecycle of private information processing. It transforms local text files into searchable vector embeddings, allowing the model to retrieve relevant context and ground its generated r

This project is a comprehensive educational curriculum and structured learning path covering the full lifecycle of large language models. It provides a guided progression through the theory, architecture, training, and deployment of these models. The curriculum includes specialized guides on transformer architecture, model training tutorials, and frameworks for designing autonomous agents. It also provides dedicated resources for studying model safety and ethics. The material covers a wide range of technical capabilities, including distributed training strategies, parameter-efficient fine-tu

DeepPavlov is a conversational AI framework and deep learning NLP library designed for building end-to-end dialogue systems and chatbots. It functions as an NLP pipeline orchestrator that allows users to compose pre-trained models and text processing components into sequential data flows for complex linguistic tasks. The system is distinguished by its ability to act as a chatbot deployment server, exposing trained conversational models as web services via REST and Socket APIs. It utilizes JSON-based pipeline configurations and dynamic variable interpolation to decouple model logic from infras

The Hugging Face Hub Python client is a library that provides programmatic access to the Hugging Face Hub, a centralized platform for hosting and collaborating on machine learning models, datasets, and demo applications. It serves as the primary SDK for interacting with the Hub's API, enabling users to download and upload models and datasets, manage repositories, authenticate via tokens or OAuth, and run inference on hosted models through a unified interface. The client distinguishes itself through a comprehensive set of capabilities that go beyond basic file transfer. It includes a CLI exten

BioGPT is a biomedical large language model and domain-specific transformer designed for processing and creating specialized medical text. It functions as a generative tool and knowledge extraction engine trained on large-scale scientific literature to produce human-like scientific prose and factual responses to queries. The project provides specialized capabilities for biomedical named entity recognition and the extraction of complex relations from unstructured medical corpora. It is designed to identify and classify biological entities through data mining and relation extraction to support

Tools for using Maluuba's news questions and answer data. The code in the repo is used to compile the dataset since it cannot be made directly available due to legal reasons.

DrQA is an open-domain question answering system that retrieves relevant documents from a large corpus and extracts specific answers to natural language questions. It is implemented as a neural network system that combines a document retrieval engine with a machine reading comprehension model. The system utilizes a two-stage pipeline architecture. A coarse-grained document retriever uses weighted word vectors to identify potential documents, while a fine-grained machine reading comprehension model identifies and extracts the exact text span containing the answer. The project also includes a

Data and code for ACL 2023 Findings: Aligning Instruction Tasks Unlocks Large Language Models as Zero-Shot Relation Extractors.

Official repository of paper "Zero-Shot Information Extraction via Chatting with ChatGPT". Please star, watch, and fork our repo for the active updates!

This project is a plugin framework and agentic workflow library designed to connect large language models to professional toolstacks. It provides a system for integrating language models with external data warehouses, CRMs, and other enterprise software to retrieve and manipulate real-time business data. The framework enables the automation of specialized professional tasks through a file-based plugin definition system. It allows for the customization of domain expertise and plugin behavior to align with internal company processes, supported by an enterprise data connector that links models t