SUP-MIMIC: A Multi-Task Clinical Diagnosis Benchmark for Evaluating LLMs' Robustness to Contradictory Evidence

Paper recorded by Signals 4 on 2026-08-30 in cs.AI. Abstract reproduced from arXiv; link to the original below.

Published 2026-08-30 on arXiv · recorded by Signals 4 on 2026-09-01

Category: cs.AI · 人工智能 · first seen 2026-09-01

Abstract

Current evaluations of large language models (LLMs) primarily focus on factual knowledge retrieval, overlooking the fundamental challenge of navigating the complex, non-bijective mappings between clinical indicators and diagnoses. Existing benchmarks fail to assess whether large language models truly possess the reasoning capability required for diagnostic ambiguity scenarios, where identical clin

Read on arXiv →