Awesome GitHub RepositoriesModel Behavior Evaluation

CleanRL is a reinforcement learning library and PyTorch framework providing a suite of reproducible implementations for online reinforcement learning algorithms. It serves as a deep reinforcement learning benchmark suite and experiment orchestrator designed for research and agent development across both discrete and continuous action spaces. The project is distinguished by its single-file algorithm implementation approach, which encapsulates each algorithm in a standalone script to eliminate complex class hierarchies. This structure is paired with a system for scheduling and executing large-s

Arize Phoenix is an LLM observability platform and evaluation framework designed to capture execution traces and monitor large language model applications. It serves as a prompt management system for versioning and testing templates, and as a self-hosted AI operations infrastructure for managing telemetry and experiments. The platform differentiates itself through a specialized embedding visualization tool used to detect data drift and optimize vector search. It provides a comprehensive evaluation suite that utilizes judge-based evaluators and ground-truth datasets to score model outputs, and

Lab is a customizable 3D platform and research testbed designed for training and testing autonomous agents using reinforcement learning. It serves as a spatial AI training simulator where agents can be evaluated through navigation and puzzle-solving tasks. The environment allows for the definition of complex layouts and task behaviors through external scripting, enabling the generation of specific challenges for AI research. It supports both automated training via standard API bindings and manual agent control to validate simulation dynamics. The system utilizes a grid-based spatial represen

Learn_Prompting ist ein pädagogisches Projekt mit Fokus auf Prompt Engineering. Es vermittelt die Prinzipien und Techniken, die erforderlich sind, um effektive Inputs zu erstellen und die Qualität von generativen KI-Outputs zu verbessern. Das Projekt deckt fortgeschrittene Prompting-Strategien ab, um Reasoning, Zuverlässigkeit und Output-Qualität zu steigern. Dies umfasst Techniken für Task-Decomposition, Chain-of-Thought-Reasoning sowie den Einsatz von Few-Shot- und Zero-Shot-Guidance. Es adressiert zudem Modellsicherheit durch das Studium von Prompt-Hacking, Schwachstellenanalyse und Privacy-Auditing, um sensible Datenlecks zu verhindern. Der Umfang erstreckt sich auf die praktische Anwendung generativer KI über diverse Medien und Workflows hinweg, einschließlich Textgenerierung, fotorealistischer Bilderstellung und audiovisueller Produktion. Es behandelt zudem die Entwicklung autonomer Agenten, KI-gestützte Programmierung und die Automatisierung von Business-Workflows für Marketing und Kommunikation. Das Projekt bietet Ressourcen für Modelloptimierung, Evaluierung und das Management von Prompt-Lifecycles innerhalb einer interaktiven Experimentierumgebung.

PhiCookBook is a technical guide and implementation framework for integrating small language models into applications. It provides instructions for deploying these lightweight models to perform reasoning, coding, and math tasks across various hardware environments and serving platforms. The project functions as a tutorial for developing intelligent AI applications by chaining prompts and code into executable sequences. It includes a framework for evaluating model behavior and calculating quality metrics to verify the accuracy and reliability of these workflows. The repository covers a broad