PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. 🏆 Best Paper Awards @ NeurIPS ML Safety Workshop 2022
PyRIT is an AI vulnerability assessment tool and security scanner designed to detect risks in large language model applications. It functions as a generative AI red teaming framework used to simulate adversarial attacks and identify weaknesses in system guardrails. The tool automates AI risk assessment by scanning generative AI components for security vulnerabilities. It utilizes automated testing and analysis to identify security gaps and prevent potential exploits through a consistent, repeatable process. The system incorporates asynchronous model orchestration to compare security postures
In this paper, we conduct the first work to propose the concept of indirect jailbreak and achieve Retrieval-Augmented Generation (RAG) via LangChain. Building on this, we further design a novel method of indirect jailbreak attack, termed Poisoned-LangChain (PLC), which leverages a poisoned…
This is the official repo of the paper "Attack Prompt Generation for Red Teaming and Defending Large Language Models" accepted to Findings of EMNLP 2023.
We present a new class of vulnerabilities and impacts stemming from "indirect prompt injection" affecting language models integrated with applications. Our demos currently span GPT-4 (Bing and synthetic apps) using ChatML, GPT-3 & LangChain based apps in addition to proof-of-concepts for attacks…
The main features of greshake/llm-security are: AI Security, Jailbreak Attack Methods.
Open-source alternatives to greshake/llm-security include: agencyenterprise/promptinject — PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the… azure/pyrit — PyRIT is an AI vulnerability assessment tool and security scanner designed to detect risks in large language model… cam-fss/jailbreak-langchain — In this paper, we conduct the first work to propose the concept of indirect jailbreak and achieve Retrieval-Augmented… chawins/pal — Chawin Sitawarin 1 Norman Mu 1 David Wagner 1 Alexandre Araujo 2. cleverhans-lab/cleverhans — Cleverhans is an adversarial machine learning library and toolkit designed to generate adversarial examples,… aatrox103/sap — This is the official repo of the paper "Attack Prompt Generation for Red Teaming and Defending Large Language Models"…