# ucsc-vlaa/vllm-safety-benchmark

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/ucsc-vlaa-vllm-safety-benchmark).**

88 stars · 5 forks · Python

## Links

- GitHub: https://github.com/UCSC-VLAA/vllm-safety-benchmark
- Homepage: https://arxiv.org/abs/2311.16101
- awesome-repositories: https://awesome-repositories.com/repository/ucsc-vlaa-vllm-safety-benchmark.md

## Topics

`adversarial-attacks` `benchmark` `datasets` `llm` `multimodal-llm` `robustness` `safety` `vision-language-model`

## Description

[ECCV 2024] Official PyTorch Implementation of "How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs"

## Tags

### Part of an Awesome List

- [Evaluation Benchmarks](https://awesome-repositories.com/f/awesome-lists/ai/evaluation-benchmarks.md) — Safety evaluation benchmark specifically for vision-language models.