Awesome GitHub RepositoriesAdversarial Input Detection

Superagent is an AI safety platform that protects applications from prompt injections, data leaks, and harmful outputs through built-in guardrails. It functions as a prompt injection detection system, data redaction tool, and red team testing tool, automatically removing personally identifiable information and protected health data from AI inputs and outputs while scanning image uploads with vision AI to detect visual prompt injection attacks before processing. The platform routes every prompt through a sequential pipeline of safety checks including injection detection, data redaction, and co

NeMo-Guardrails is a toolkit for adding programmable safety constraints and dialogue boundaries to large language model conversational systems. It functions as security middleware that intercepts inputs and outputs to block prompt injections, jailbreaks, and sensitive data leaks, while providing a conversational dialogue manager to define structured interaction flows through configuration files. The framework includes a hallucination filter to screen model outputs for factual accuracy and a specialized modeling language for defining conversational flows and constraints. It provides capabiliti

The Adversarial Robustness Toolbox (ART) is an open-source library that provides a unified framework for evaluating, defending, and certifying machine learning models against adversarial threats. It wraps models from any framework behind a common estimator interface, enabling composable pipelines for attack generation, defense application, robustness certification, and privacy auditing across evasion, poisoning, and extraction threats. The library distinguishes itself by covering the full adversarial ML security lifecycle within a single toolkit. It supports gradient-based adversarial example

Helicone is an AI gateway and observability platform designed to intercept, manage, and monitor interactions with large language models. By acting as a reverse-proxy, it provides a centralized layer for routing requests across multiple AI providers, allowing developers to maintain consistent application logic while gaining deep visibility into model performance, usage, and costs. The platform distinguishes itself through a robust suite of traffic management and prompt engineering tools. It enables policy-driven control, including automatic failover between providers, rate limiting, and edge-b

agent-governance-toolkit 是一个用于执行安全策略、管理零信任身份以及沙箱化自主 AI 代理执行的框架。它提供了一个治理层，旨在通过使用安全策略引擎、加密身份管理和运行时执行沙箱来控制代理的行为。 该项目通过多级特权环系统和加密身份网格脱颖而出，该网格保护自主实体之间的通信。它实现了基于衰减的信任评分机制来跟踪实体可靠性，并利用哈希链式、防篡改审计日志来维护可验证的执行历史。 该工具包涵盖了广泛的能力领域，包括防御注入攻击的提示词安全、针对监管标准的自动化合规性映射，以及使用 Saga 模式的自主工作流编排。它还具有用于跟踪健康状况和支出限额的舰队监控，以及用于限制未经授权资源访问的工具执行沙箱。 提供了一个命令行界面，用于执行控制信号、验证治理策略以及管理扩展的安装。

LLM Guard is a security firewall and guardrail framework designed to scan and sanitize inputs and outputs for large language models. It functions as a proxy gateway and security layer to block prompt injections, toxicity, and sensitive data leakage while ensuring that model interactions remain compliant with organizational policies. The system distinguishes itself through a modular scanner pipeline that utilizes local model orchestration to eliminate external network dependencies. It supports real-time security filtering via streaming chunk analysis and implements a fail-fast execution model