Paper recorded by Signals 4 on 2026-10-26 in cs.LG. Abstract reproduced from arXiv; link to the original below.
Published 2026-10-26 on arXiv · recorded by Signals 4 on 2026-10-27
Category: cs.LG · 机器学习 · first seen 2026-10-27
Preference optimization can learn spurious correlations, and augmenting the preference data with ties (equally preferred responses) can reduce spurious reliance. However, ties differ in their corrective effect, making their selection for augmentation important. For log-linear policies, we derive each tie's influence on spurious reliance and show that taking the largest predicted reductions is firs