Open-source alternatives to BioGPT

This project is a standardized framework for benchmarking large language models across a wide range of academic and reasoning datasets. It provides a platform for executing automated evaluation tasks to measure model accuracy and performance, ensuring consistent assessment through a structured configuration schema. The framework distinguishes itself by incorporating a dedicated utility for data decontamination, which identifies and removes overlapping training samples from evaluation sets to prevent data leakage. It also features a flexible task builder that allows users to define custom benc

This project serves as a comprehensive, static directory of external resources dedicated to the study and application of large language models. It functions as a centralized discovery point for developers and researchers, aggregating foundational academic papers, technical documentation, and specialized tools within a structured, version-controlled knowledge base. The repository distinguishes itself through a multi-level classification system that organizes diverse technical domains, ranging from model training frameworks and inference optimization to AI safety and hallucination detection. By

This project is a suite of utilities for creating synthetic training data, performing model fine-tuning, and verifying output quality through evaluation frameworks. It provides a toolkit for optimizing pre-trained large language models to improve performance on specific tasks. The system includes a synthetic dataset generator that creates diverse input-output training pairs from task descriptions. It also features a system prompt generator to produce the behavioral constraints and messages required to guide a fine-tuned model. The toolkit covers a complete workflow for model refinement, incl

This project is a comprehensive ecosystem of frameworks, toolkits, and datasets designed to evaluate model vulnerabilities and analyze jailbreak patterns. It serves as an adversarial testing framework and research toolkit for measuring the effectiveness of safety guardrails in large language models. The system includes a library of real-world prompt injection datasets harvested from social media to study bypass strategies. It provides specialized tools for semantic attack analysis and prompt visualization, allowing for the mapping of relationships between adversarial prompts to discover commo

MindSearch is an LLM-based multi-agent search engine that decomposes complex user questions into targeted sub-queries and routes each to a specialized agent for parallel investigation. The system orchestrates multiple agents through a large language model, coordinating their tasks and interpreting search results to produce coherent answers from multiple sources. The project provides a configurable search backend interface that allows switching between Google, DuckDuckGo, Brave, and Bing search APIs by updating a configuration attribute. It includes a terminal-based debug interface for testing

R1-V is a toolset for the development of multimodal models, providing a low-cost training environment designed to optimize the reasoning and feedback loops of large vision-language models. It integrates a training framework, fine-tuning pipelines, and performance evaluation tools. The project features a reinforcement learning framework that improves visual reasoning and generalization by rewarding correct outputs based on visual verification. It also includes a supervised fine-tuning pipeline for customizing vision-language models to specific tasks using labeled datasets and configuration fil

VLMEvalKit is a vision-language model evaluation framework and inference engine designed to run standardized benchmarks and measure model accuracy across diverse visual datasets. It serves as a multimodal model benchmark and performance toolkit for calculating metrics and comparing model responses. The toolkit includes a specialized visual reasoning evaluator that uses adversarial samples to distinguish actual image understanding from reliance on language patterns. It also provides capabilities for image generation evaluation, testing a model's ability to create or modify visuals based on tex

This project is a language model evaluation framework and benchmarking tool designed to measure the accuracy and performance of models across diverse datasets. It provides a system for implementing model-based graders, running standardized tests for mathematical reasoning, coding, and factuality, and calculating quantified performance metrics such as precision, recall, F1 scores, and pass-at-k. The framework utilizes model-based grading and rubrics to validate response quality against expert-defined criteria. It includes a multi-model benchmarking loop and a model-agnostic API interface to co

This project is a collection of utilities designed for machine learning experiment tracking, data versioning, and the observability of large language model applications. It provides a client for recording hyperparameters and metrics during training to visualize performance trends and compare different model versions. The tool includes a model evaluation framework that uses custom scorers and automated judges to assess the quality of generated text outputs. It also provides observability tools to monitor and debug the execution flow and runtime behavior of language model applications. The sys

This project is a collection of scripts and workflows for training, fine-tuning, and deploying large language models using the Hugging Face Transformers toolkit. It functions as a distributed training framework, a library for natural language processing task implementations, and a system for building retrieval-augmented generation chatbots. The repository includes specialized tools for model optimization, such as a Bayesian hyperparameter optimizer for automatically tuning model settings. It provides implementations for scaling model training across multiple graphics processors using data par

Gorilla is a foundational infrastructure framework for large language model function calling. It provides a system for training, evaluating, and executing the translation of natural language instructions into accurate API calls and executable code. The project integrates a structured API documentation index, a fine-tuning pipeline for model adaptation, and a secure sandboxed action runtime for executing model-generated commands. The framework distinguishes itself through a specialized evaluation benchmark suite that measures the accuracy, cost, and latency of function calls. It includes tools

This project is a comprehensive library for transfer learning and domain adaptation in computer vision. It serves as a framework for aligning feature distributions between source and target datasets, a toolkit for domain generalization, and a library for semi-supervised learning using small labeled datasets and large unlabeled sets. The library provides specialized capabilities for unsupervised domain adaptation, including the use of adversarial networks, discrepancy-based architectures, and image-to-image translation to reduce distribution mismatch. It also includes tools for domain generali

Reor is a local AI knowledge management application that stores, links, and searches personal notes using large language models and vector embeddings entirely on the user's device. It functions as a private AI note assistant, keeping all data and processing local for full privacy without relying on external cloud services. The application integrates with Ollama to manage the lifecycle of local LLMs and embedding models, handling downloads, updates, and execution. Notes are imported from markdown files, preserving existing file structure, and are automatically linked through vector-similarity

This project is an implementation of the ALBERT language model architecture, providing a framework for training and evaluating transformer-based text classifiers and similarity models. It specifically includes pre-trained assets and tools optimized for generating semantic embeddings and representations of Chinese text. The framework distinguishes itself through tools for converting heavy language model checkpoints into lightweight formats to enable low-latency inference on mobile devices. It utilizes specific weight reduction techniques, including cross-parameter sharing and factorized embedd

This project is a collection of educational resources and implementation frameworks providing deep learning model recipes, code samples, and step-by-step guides for computer vision tasks. It organizes complex workflows into modular recipes and implementation guides to facilitate the building of image and video analysis models. The framework focuses on specialized vision capabilities, including an image similarity framework for fast retrieval and re-ranking, human pose estimation, and video action recognition. It also provides specific tools for crowd density estimation and document image clea

SynapseML is an Apache Spark machine learning library designed for building and scaling machine learning workflows and data pipelines across distributed clusters. It serves as a distributed machine learning pipeline framework and a distributed inference engine for executing hardware-accelerated predictions and deep learning tasks on large-scale datasets. The project functions as a cloud AI integration layer, allowing users to apply pretrained artificial intelligence services for text, vision, and speech within distributed pipelines. It also includes a dedicated suite of tools for distributed

This project is a named entity recognition framework and TensorFlow-based natural language processing model. It provides a pipeline for adapting pre-trained language models to specific entity recognition and text classification tasks. The system implements a sequence labeling architecture that combines transformer-based embeddings with bidirectional sequence modeling and conditional random field decoding. It includes tools for fine-tuning model weights and training the network to identify and categorize entities within unstructured text. The framework also includes a client-server architectu

nlp-recipes is a collection of implementation guides and reference templates for applying natural language processing techniques to real-world tasks. It provides standardized workflows and code examples for developing NLP pipelines, from dataset preparation and model training to performance evaluation. The project focuses on the practical application of transformer-based models, offering patterns for fine-tuning pretrained architectures for tasks such as text classification, named entity recognition, and question answering. It also includes a toolkit for model interpretability, allowing users

OpenChatKit is a training and inference toolkit for large language models. It provides a comprehensive set of tools for managing the model lifecycle, including a fine-tuning pipeline, a model weight converter, and a command-line interface for interacting with conversational agents. The toolkit features a framework for retrieval augmented generation, allowing models to incorporate relevant context from external vector indices. It also includes utilities for converting trained model checkpoints into formats compatible with standard inference libraries. The project covers conversational AI trai

OpenCompass is an open-source framework for standardized benchmarking of large language models. It provides a configurable evaluation pipeline that supports both objective and subjective assessment, using a dual-engine architecture to handle closed-form answer comparison and open-ended response rating. The framework is designed as a modular platform where datasets, models, and metrics are composed through declarative YAML configuration files. The framework distinguishes itself through its extensible model integration layer, which supports custom models, HuggingFace models, and third-party API

Orange3 is a visual data mining platform that provides an interactive canvas for building data analysis workflows without writing code. At its core, it offers a widget-based visual programming environment where users connect configurable components to perform data preprocessing, machine learning model training, statistical evaluation, and interactive visualization. The platform is built on NumPy-backed data tables with domain descriptors that define variable names, types, and roles, and includes a lazy SQL query proxy for working with database tables without loading all data into memory. The

Open-r1 is a framework designed for the large-scale training, distillation, and optimization of language models focused on complex reasoning and programming tasks. It provides a comprehensive suite of tools for managing distributed training jobs across multi-node clusters, enabling the development of high-performance models through reinforcement learning and supervised fine-tuning. The project distinguishes itself by integrating secure, containerized code execution environments directly into the training and evaluation lifecycle. By allowing models to run and verify code snippets against test

The codes and dataset will be released soon!

This is the codebase of the paper: ERA-CoT: Improving Chain-of-Thought through Entity Relationship Analysis.

The original implementation of the paper. You can cite the paper as below.