Incremental Pooled LLM Evaluation for Cost-Effective Retrieval Model Selection

Paper recorded by Signals 4 on 2026-09-02 in cs.CL. Abstract reproduced from arXiv; link to the original below.

Published 2026-09-02 on arXiv · recorded by Signals 4 on 2026-09-03

Category: cs.CL · 自然语言处理 · first seen 2026-09-03

Abstract

Selecting a retrieval model for a production RAG system requires reliable comparative evaluation, but obtaining relevance judgments at scale is expensive and difficult to repeat as new candidate systems arrive. We study pooled LLM evaluation, in which an LLM judges the union of documents retrieved by the current set of candidate systems, and the pool is then expanded incrementally as new systems a

Read on arXiv →