How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
MatchZoo is a deep learning framework designed for building, training, and evaluating neural networks that determine the relevance and similarity between pairs of textual inputs. It serves as a research platform for neural information retrieval, specifically supporting the development of models for document retrieval, question answering, and ranking tasks. The framework utilizes declarative architecture composition to define complex neural network structures. It includes automated hyper-parameter resolution to populate missing configuration parameters before model compilation and uses callbac
OpenCompass is an open-source framework for standardized benchmarking of large language models. It provides a configurable evaluation pipeline that supports both objective and subjective assessment, using a dual-engine architecture to handle closed-form answer comparison and open-ended response rating. The framework is designed as a modular platform where datasets, models, and metrics are composed through declarative YAML configuration files. The framework distinguishes itself through its extensible model integration layer, which supports custom models, HuggingFace models, and third-party API
Opik is an observability and evaluation platform designed for generative AI applications and agentic workflows. It provides a centralized environment for tracing execution flows, managing prompt templates, and monitoring production performance, allowing teams to gain visibility into complex model interactions and tool usage without requiring manual application code changes. The platform distinguishes itself through its integrated approach to the AI development lifecycle, combining distributed trace instrumentation with automated evaluation frameworks. It supports model-as-a-judge scoring, syn
GAOKAO-Bench is an evaluation framework that utilizes GAOKAO questions as a dataset to evaluate large language models.
Official github repo for C-Eval, a Chinese evaluation suite for foundation models [NeurIPS 2023]
The main features of hkust-nlp/ceval are: Model Evaluation and Benchmarking, Natural Language Processing.
Projects with overlapping indexed features include: ntmc-community/matchzoo — MatchZoo is a deep learning framework designed for building, training, and evaluating neural networks that determine… openlmlab/gaokao-bench — GAOKAO-Bench is an evaluation framework that utilizes GAOKAO questions as a dataset to evaluate large language models. comet-ml/opik — Opik is an observability and evaluation platform designed for generative AI applications and agentic workflows. It… open-compass/opencompass — OpenCompass is an open-source framework for standardized benchmarking of large language models. It provides a… dusty-nv/jetson-inference — jetson-inference is a set of libraries and tools for executing optimized deep learning models on embedded GPU… facebookresearch/parlai — ParlAI is a conversational AI research framework designed for training, evaluating, and sharing dialogue models using…