LLM safety evaluation and red-teaming tool
For jailbreak ai, the first results are giskard-ai/giskard-oss (Giskard is an open-source Python framework and evaluation suite that provides automated red-teaming, prompt injection testing, and vulnerability scanning specifically tailored for assessing the safety and security risks of large language models), llm-attacks/llm-attacks (This Python-based repository provides tools for generating adversarial prompts and testing large language model vulnerabilities, matching the core requirements for LLM safety evaluation and automated red-teaming) and verazuo/jailbreak_llms (This repository provides a comprehensive framework and dataset ecosystem specifically designed for adversarial testing, jailbreak research, and evaluating safety vulnerabilities in large language models). promptfoo/promptfoo and giskard-ai/giskard round out the shortlist. Compare the match explanations and check the project documentation against your requirements.
We curate open-source GitHub repositories matching “jailbreak ai”. Results are ranked by relevance to your query — pick filters below to narrow, or refine with AI.
Results for “jailbreak ai”
Find the best repos with AI.We'll search the best matching repositories with AI.
- giskard-ai/giskard-oss
Giskard-AI/giskard-oss
5,467View on GitHubGiskard is an AI quality assurance suite and evaluation framework designed to measure the performance, bias, and security risks of large language models and AI agents. It functions as a vulnerability scanner to detect security flaws and performance regressions. The project provides automated red-teaming and adversarial testing workflows. These tools generate prompt-injection probes and adversarial attacks based on system descriptions to identify security gaps and vulnerabilities. The platform covers AI agent auditing and RAG quality validation, using knowledge-base grounding and synthetic da
Giskard is an open-source Python framework and evaluation suite that provides automated red-teaming, prompt injection testing, and vulnerability scanning specifically tailored for assessing the safety and security risks of large language models.
PythonAdversarial Test AutomationPrompt Injection TestingAI Prompt Injection Vulnerabilities - llm-attacks/llm-attacks
llm-attacks/llm-attacks
4,509View on GitHubThis repository provides tools and methodologies for studying adversarial attacks on large language models. It focuses on understanding how carefully crafted inputs can manipulate or bypass the safety mechanisms of LLMs, enabling researchers to probe model vulnerabilities and improve their robustness. The project covers techniques for generating adversarial prompts, evaluating model responses under attack conditions, and analyzing the effectiveness of different attack strategies.
This Python-based repository provides tools for generating adversarial prompts and testing large language model vulnerabilities, matching the core requirements for LLM safety evaluation and automated red-teaming.
PythonAdversarial Suffix GenerationAdversarial Suffix Optimization - verazuo/jailbreak_llms
verazuo/jailbreak_llms
3,563View on GitHubThis project is a comprehensive ecosystem of frameworks, toolkits, and datasets designed to evaluate model vulnerabilities and analyze jailbreak patterns. It serves as an adversarial testing framework and research toolkit for measuring the effectiveness of safety guardrails in large language models. The system includes a library of real-world prompt injection datasets harvested from social media to study bypass strategies. It provides specialized tools for semantic attack analysis and prompt visualization, allowing for the mapping of relationships between adversarial prompts to discover commo
This repository provides a comprehensive framework and dataset ecosystem specifically designed for adversarial testing, jailbreak research, and evaluating safety vulnerabilities in large language models.
Jupyter NotebookPrompt Injection TechniquesJailbreak Prompts - promptfoo/promptfoo
promptfoo/promptfoo
10,529View on GitHubPromptfoo is an evaluation framework designed for testing, benchmarking, and red-teaming language models and agentic workflows. It provides a unified environment to run prompts against multiple providers, allowing developers to systematically validate model outputs against objective assertions, semantic similarity metrics, and custom grading rubrics. The platform distinguishes itself through a provider-agnostic execution layer and a stateful orchestrator capable of simulating multi-turn conversations and complex tool-use trajectories. It includes a dedicated adversarial mutation pipeline that
Promptfoo is an evaluation and red-teaming framework that tests large language models for prompt injections and security vulnerabilities with automated assertions and adversarial payloads.
TypeScriptAdversarial Test AutomationPrompt Injection Testing - giskard-ai/giskard
Giskard-AI/giskard
5,434View on GitHubGiskard is an evaluation framework, testing library, and quality monitoring system for large language models and AI agents. It serves as a toolkit for quantifying model performance and reliability, providing specialized capabilities for validating retrieval-augmented generation pipelines. The project distinguishes itself through an automated red teaming tool and security scanner designed to identify vulnerabilities, prompt injections, and safety risks. It utilizes adversarial probing and synthetic edge case generation to quantify model robustness and detect information disclosure. The platfo
Giskard is an evaluation framework and security scanner for large language models that features automated red-teaming, prompt injection testing, and adversarial probing, making it a comprehensive tool for this specific category.
PythonAI Prompt Injection Vulnerabilities - nvidia/garak
NVIDIA/garak
8,114View on GitHubGarak is an AI model evaluation tool and vulnerability scanner designed for red teaming large language models and auditing the security of retrieval-augmented generation pipelines. It identifies behavioral weaknesses, such as jailbreaks, hallucinations, and data leakage, by simulating adversarial attacks and executing automated testing vectors. The framework utilizes an adaptive probing loop where prompts can react to previous model behavior and be modified in flight via middleware. To ensure consistent analysis, it employs a provider-agnostic interface to interact with various model APIs and
Garak is a Python-based LLM security evaluation and red-teaming framework that performs automated vulnerability scanning, prompt injection testing, and adversarial probing, perfectly matching the required capabilities.
PythonAdversarial TestingLLM Vulnerability ScannersAdaptive Probing Loops - jailbreakbench/jailbreakbench
JailbreakBench/jailbreakbench
JailbreakBench -->
JailbreakBench is a Python-based evaluation benchmark and repository designed for analyzing and probing large language models against jailbreak exploits, though it is narrower on standardized benchmarking than automated red-teaming payload generation.
PythonEvaluation Benchmarks - microsoft/promptbench
microsoft/promptbench
2,808View on GitHubA unified evaluation framework for large language models
Promptbench is a Python evaluation framework designed for assessing large language models against adversarial attacks like prompt injection, making it well-suited for safety testing even though it focuses more on benchmarking than automated multi-turn red-teaming.
PythonAdversarial TestingEvaluation FrameworksMachine Learning Research - utkusen/promptmap
utkusen/promptmap
1,208View on GitHub` O O o.-. Humans, Do Not Resist! \|/ ,-'-.() / /\, / /\|.-.| / --O-- (.--""" /\ /\ | ( o.o ) / Utku Sen's /|\ -'--' / /| | - | / | | | \ /| | | ' \ '/ \ ' \| ' \ | ' \/ | ' \ ) | / | | | ./| \/||| ./\|||\,| .// / / |-----| || || || |_| ``
This Python-based tool focuses on adversarial testing and prompt injection exploits against language models, making it a relevant option for security evaluation despite its sparse description.
PythonAdversarial TestingAI SecuritySecurity and Safety - controllability/jailbreak-evaluation
controllability/jailbreak-evaluation
The jailbreak-evaluation is an easy-to-use Python package for language model jailbreak evaluation. The jailbreak-evaluation is designed for comprehensive and accurate evaluation of language model jailbreak attempts. Currently, jailbreak-evaluation support evaluating a language model jailbreak…
This repository provides a Python package specifically designed for evaluating language model jailbreak attempts, matching the core safety evaluation intent even though it lacks explicit mention of automated red-teaming or payload generation.
PythonEvaluation Benchmarks - microsoft/ai-red-teaming-playground-labs
microsoft/AI-Red-Teaming-Playground-Labs
1,834View on GitHubThis repository provides a playground environment specifically designed for AI red-teaming and prompt-injection testing, though its thin description and TypeScript-focused setup lack the full scope of a dedicated Python testing framework.
TypeScriptai-red-teamai-red-teamingprompt-injection - msoedov/agentic_security
msoedov/agentic_security
1,911View on GitHubThis Python-based AI red-teaming kit and vulnerability scanner provides automated scanning and fuzzer capabilities for testing large language models, covering much of the requested safety evaluation scope.
Pythonagent-frameworkagent-securityai-red-team
Compare the top 10 at a glance
| Repository | Stars | Language | License | Last push |
|---|---|---|---|---|
| giskard-ai/giskard-oss | 5.5K | Python | Apache-2.0 | |
| llm-attacks/llm-attacks | 4.5K | Python | mit | |