How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
Garak is an AI model evaluation tool and vulnerability scanner designed for red teaming large language models and auditing the security of retrieval-augmented generation pipelines. It identifies behavioral weaknesses, such as jailbreaks, hallucinations, and data leakage, by simulating adversarial attacks and executing automated testing vectors. The framework utilizes an adaptive probing loop where prompts can react to previous model behavior and be modified in flight via middleware. To ensure consistent analysis, it employs a provider-agnostic interface to interact with various model APIs and
A curation of awesome tools, documents and projects about LLM Security.
Cleverhans is an adversarial machine learning library and toolkit designed to generate adversarial examples, incorporate them into training loops, and benchmark the resilience of machine learning models. It provides a gradient-based attack framework for constructing both white-box and black-box attacks to identify model misclassifications. The project includes capabilities for model robustness benchmarking, allowing users to evaluate and verify how models resist evasion attacks and malicious input perturbations. It also facilitates adversarial training to increase a model's resistance to pert
` O O o.-. Humans, Do Not Resist! \|/ ,-'-.() / /\, / /\|.-.| / --O-- (.--""" /\ /\ | ( o.o ) / Utku Sen's /|\ -'--' / /| | - | / | | | \ /| | | ' \ '/ \ ' \| ' \ | ' \/ | ' \ ) | / | | | ./| \/||| ./\|||\,| .// / / |-----| || || || |_| ``
The main features of utkusen/promptmap are: Adversarial Testing, AI Security, Security and Safety.
Projects with overlapping indexed features include: nvidia/garak — Garak is an AI model evaluation tool and vulnerability scanner designed for red teaming large language models and… confident-ai/deepteam. corca-ai/awesome-llm-security — A curation of awesome tools, documents and projects about LLM Security. cranot/chatbot-injections-exploits — ChatBot Injection and Exploit Examples: A Curated List of Prompt Engineer Commands - ChatGPT. greshake/llm-security — We present a new class of vulnerabilities and impacts stemming from "indirect prompt injection" affecting language… cleverhans-lab/cleverhans — Cleverhans is an adversarial machine learning library and toolkit designed to generate adversarial examples,…