MMSegmentation is an open-source semantic segmentation toolbox built on PyTorch that provides a modular, configurable framework for building, training, evaluating, and deploying segmentation models. At its core, it offers a config-driven pipeline that assembles training, evaluation, and inference workflows by parsing hierarchical configuration files, with a modular component registry that enables plug-and-play composition of neural network modules, optimizers, datasets, and metrics. The framework supports the full model lifecycle through a unified runner interface that controls training, testi
ParlAI is a conversational AI research framework designed for training, evaluating, and sharing dialogue models using a unified interface for datasets and agents. It functions as a PyTorch-based training platform and a dialogue data collection system, providing a centralized model zoo for the distribution of versioned pretrained agents. The project distinguishes itself through a knowledge-grounded retrieval system that combines dense and sparse indexing to ground responses in external information. It also provides a comprehensive infrastructure for gathering human-AI interaction data via inte
This project is a language model evaluation framework and benchmarking tool designed to measure the accuracy and performance of models across diverse datasets. It provides a system for implementing model-based graders, running standardized tests for mathematical reasoning, coding, and factuality, and calculating quantified performance metrics such as precision, recall, F1 scores, and pass-at-k. The framework utilizes model-based grading and rubrics to validate response quality against expert-defined criteria. It includes a multi-model benchmarking loop and a model-agnostic API interface to co
[ICML 2024] TrustLLM: Trustworthiness in Large Language Models
Las características principales de howiehwong/trustllm son: Model Evaluation and Benchmarking.
Las alternativas de código abierto para howiehwong/trustllm incluyen: open-mmlab/mmsegmentation — MMSegmentation is an open-source semantic segmentation toolbox built on PyTorch that provides a modular, configurable… packtpublishing/llm-engineers-handbook — This project is an educational resource and engineering guide for building, deploying, and optimizing large language… facebookresearch/parlai — ParlAI is a conversational AI research framework designed for training, evaluating, and sharing dialogue models using… infrasys-ai/aiinfra. openai/simple-evals — This project is a language model evaluation framework and benchmarking tool designed to measure the accuracy and… facebookresearch/slowfast — SlowFast is a PyTorch video understanding framework and spatiotemporal neural network library. It serves as a toolset…
