Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Models

Paper recorded by Signals 4 on 2026-08-30 in cs.CV. Abstract reproduced from arXiv; link to the original below.

Published 2026-08-30 on arXiv · recorded by Signals 4 on 2026-09-01

Category: cs.CV · 计算机视觉 · first seen 2026-09-01

Abstract

We propose a societal bias evaluation method for large vision-language models (LVLMs) in the era of strong safety guardrails. Existing benchmarks rely on prompts that ask models to infer attributes of people in images (e.g., "Is this person a CEO or a secretary?"). However, we find that LVLMs with strong guardrails, such as GPT and Claude, often refuse these prompts, making evaluations unreliable.

Read on arXiv →