Awesome GitHub RepositoriesModel Behavior Evaluation

CleanRL is a reinforcement learning library and PyTorch framework providing a suite of reproducible implementations for online reinforcement learning algorithms. It serves as a deep reinforcement learning benchmark suite and experiment orchestrator designed for research and agent development across both discrete and continuous action spaces. The project is distinguished by its single-file algorithm implementation approach, which encapsulates each algorithm in a standalone script to eliminate complex class hierarchies. This structure is paired with a system for scheduling and executing large-s

Arize Phoenix is an LLM observability platform and evaluation framework designed to capture execution traces and monitor large language model applications. It serves as a prompt management system for versioning and testing templates, and as a self-hosted AI operations infrastructure for managing telemetry and experiments. The platform differentiates itself through a specialized embedding visualization tool used to detect data drift and optimize vector search. It provides a comprehensive evaluation suite that utilizes judge-based evaluators and ground-truth datasets to score model outputs, and

Lab is a customizable 3D platform and research testbed designed for training and testing autonomous agents using reinforcement learning. It serves as a spatial AI training simulator where agents can be evaluated through navigation and puzzle-solving tasks. The environment allows for the definition of complex layouts and task behaviors through external scripting, enabling the generation of specific challenges for AI research. It supports both automated training via standard API bindings and manual agent control to validate simulation dynamics. The system utilizes a grid-based spatial represen

Learn_Prompting es un proyecto educativo centrado en la ingeniería de prompts, que proporciona los principios y técnicas necesarios para elaborar entradas efectivas y mejorar la calidad de las salidas de la IA generativa. El proyecto cubre estrategias avanzadas de prompting para mejorar el razonamiento, la fiabilidad y la calidad de la salida. Esto incluye técnicas para la descomposición de tareas, razonamiento de cadena de pensamiento (chain-of-thought) y el uso de guías few-shot y zero-shot. También aborda la seguridad del modelo mediante el estudio de prompt hacking, análisis de vulnerabilidades y auditorías de privacidad para prevenir fugas de datos sensibles. El alcance se extiende a la aplicación práctica de la IA generativa en diversos medios y flujos de trabajo, incluyendo generación de texto, creación de imágenes fotorrealistas y producción audiovisual. Además, cubre el desarrollo de agentes autónomos, programación asistida por IA y la automatización de flujos de trabajo empresariales para marketing y comunicaciones. El proyecto proporciona recursos para la optimización de modelos, evaluación y gestión de ciclos de vida de prompts dentro de un entorno de experimentación interactivo.

PhiCookBook is a technical guide and implementation framework for integrating small language models into applications. It provides instructions for deploying these lightweight models to perform reasoning, coding, and math tasks across various hardware environments and serving platforms. The project functions as a tutorial for developing intelligent AI applications by chaining prompts and code into executable sequences. It includes a framework for evaluating model behavior and calculating quality metrics to verify the accuracy and reliability of these workflows. The repository covers a broad