How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. 🏆 Best Paper Awards @ NeurIPS ML Safety Workshop 2022
Security toolkit for AI agents. Scan your machine for dangerous skills and MCP configs, monitor for supply chain attacks, test prompt injection resistance, and audit live MCP servers for tool poisoning.
PyRIT is an AI vulnerability assessment tool and security scanner designed to detect risks in large language model applications. It functions as a generative AI red teaming framework used to simulate adversarial attacks and identify weaknesses in system guardrails. The tool automates AI risk assessment by scanning generative AI components for security vulnerabilities. It utilizes automated testing and analysis to identify security gaps and prevent potential exploits through a consistent, repeatable process. The system incorporates asynchronous model orchestration to compare security postures
This is the official repo of the paper "Attack Prompt Generation for Red Teaming and Defending Large Language Models" accepted to Findings of EMNLP 2023.
Official repo for GPTFUZZER : Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts
The main features of sherdencooper/gptfuzz are: Jailbreak Attack Methods, Red Teaming and Security.
Projects with overlapping indexed features include: agencyenterprise/promptinject — PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the… agentseal/agentseal — Security toolkit for AI agents. Scan your machine for dangerous skills and MCP configs, monitor for supply chain… azure/pyrit — PyRIT is an AI vulnerability assessment tool and security scanner designed to detect risks in large language model… cam-fss/jailbreak-langchain — In this paper, we conduct the first work to propose the concept of indirect jailbreak and achieve Retrieval-Augmented… chawins/pal — Chawin Sitawarin 1 Norman Mu 1 David Wagner 1 Alexandre Araujo 2. aatrox103/sap — This is the official repo of the paper "Attack Prompt Generation for Red Teaming and Defending Large Language Models"…