Query Timing Produces Opposite Positional Biases Between LLMs and Humans

arXiv:2608.123872026-08-14

什么时候让AI做判断,会导致它出现和人类完全相反的偏见

早前研究发现,人类如果每看完一条证据就被问一次判断,会更容易被最后出现的证据影响,产生近因偏差;但如果只在最后统一问一次,这种偏差就消失了。这篇论文对GPT、Claude等大语言模型做了同样的实验,结果发现规律正好相反。模型在逐条被追问时几乎没有偏差,但在最后才被统一问一次时,反而表现出明显偏向最后证据的近因偏差,而且这种偏差在较新的模型上更严重。

论文原文摘要（英文）

Positional biases such as recency and primacy effects have been documented in large language models (LLMs), yet the underlying mechanism by which these models make their evaluations remains poorly understood. Both primacy and recency biases have been observed in human judgments in response to evidence, but recent work suggest that \emph{when} the listener updates their beliefs -- during the presentation of evidence or only at the end -- influences the presence of such effects. We investigate whether a similar phenomenon holds for LLMs, finding divergence from human behavior. These biases are more exacerbated in newer models compared to their predecessors.

作者 · Jasin Cekinmez, Addison J. Wu, Thomas L. Griffiths