Everything K-culture — comebacks to K-beauty, straight to your inboxGet it in your inbox

METAL MEDIA

Same Facts, Different Updates: Inference Setup Shapes LLM Behavior in Medical Allocation

arXiv:2608.181082026-08-20

Asking an AI the same medical question twice gives different answers depending on how you ask

Researchers tested four language models (GPT-5.2, GPT-5-mini, DeepSeek V4 Flash, Kimi K2.5) on a task where the model assigns probabilities for prioritizing one of two patients for a medical resource, then adds one extra sentence with information like income or caregiver status and asks again. Comparing cases where the model could still see its own first answer in the conversation versus cases where it was asked completely fresh, nearly half of the tested combinations showed different, often opposite, shifts in probability. Attributes like caregiver status, low income, and parenthood pushed probability toward the clinically weaker patient, and this effect grew stronger when the wording was made more emotionally charged.

METAL MEDIA explanatory visual

Asking an AI the same medical question twice gives different answers depending on how you ask

  1. 01Four models were given two patients' treatment success rates and expected QALYs (quality-adjusted life years), asked to assign priority probabilities, then given one added sentence about income, race, caregiver status, or similar attributes and asked again
  2. 02The study compared re-asking the same question within the same conversation (model sees its prior answer) versus asking independently with no memory of the first answer, isolating the effect of the interaction setup itself
  3. 0316 of 32 model-attribute combinations showed statistically significant differences between the two setups, with 11 showing a larger shift when the prior answer stayed in context
  4. 04Caregiver status, low-income background, and parenthood consistently pushed probability toward the clinically weaker patient across most models, while sex produced no effect in any model and age produced a reversed effect
  5. 05Rewording information in a more morally charged way (e.g., 'disadvantaged' instead of 'lower-income') pushed flip rates close to 1.0 for GPT-5-mini, and smaller clinical differences between patients made models more susceptible to this non-clinical information
An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.

What they did

  1. Four models were given two patients' treatment success rates and expected QALYs (quality-adjusted life years), asked to assign priority probabilities, then given one added sentence about income, race, caregiver status, or similar attributes and asked again
  2. The study compared re-asking the same question within the same conversation (model sees its prior answer) versus asking independently with no memory of the first answer, isolating the effect of the interaction setup itself
  3. 16 of 32 model-attribute combinations showed statistically significant differences between the two setups, with 11 showing a larger shift when the prior answer stayed in context
  4. Caregiver status, low-income background, and parenthood consistently pushed probability toward the clinically weaker patient across most models, while sex produced no effect in any model and age produced a reversed effect
  5. Rewording information in a more morally charged way (e.g., 'disadvantaged' instead of 'lower-income') pushed flip rates close to 1.0 for GPT-5-mini, and smaller clinical differences between patients made models more susceptible to this non-clinical information
Figure 1: Overview of the study design. (1) Experimental cells vary severity S, clinical gap G, information content I, and framing D. (2) Paired-inference compares a baseline answer with an in-conversation social-context update; independent-inference uses independent calls. (3) Outcomes are the paired shift Δi=pB,i(2)−pB,i(1), mean shift, and flip rate.
Figure 1: Overview of the study design. (1) Experimental cells vary severity S, clinical gap G, information content I, and framing D. (2) Paired-inference compares a baseline answer with an in-conversation social-context update; independent-inference uses independent calls. (3) Outcomes are the paired shift Δi=pB,i(2)−pB,i(1), mean shift, and flip rate.
Figure 2: Independent- versus Paired-Inference by Model. Paired versus independent comparison by model and attribute, reporting the difference in mean probability shift toward Person B. Paired estimates measure within-conversation updating after the model has already answered. Independent estimates compare the independent one-shot baseline against the one-shot response with new contrastive patient attributes.
Figure 2: Independent- versus Paired-Inference by Model. Paired versus independent comparison by model and attribute, reporting the difference in mean probability shift toward Person B. Paired estimates measure within-conversation updating after the model has already answered. Independent estimates compare the independent one-shot baseline against the one-shot response with new contrastive patient attributes.
Figure 3: Attribute Comparison by Model (Pooled over Severity and Expected-QALY Gap), Absolute Shift. Attribute comparison by model, pooled over severity and expected-QALY gap, using mean paired probability shift pB(2)−pB(1). Each row is one attribute; each colored marker is one model’s point estimate, with horizontal bars showing 95% t-intervals on the run-level paired shift. The vertical reference line marks zero.
Figure 3: Attribute Comparison by Model (Pooled over Severity and Expected-QALY Gap), Absolute Shift. Attribute comparison by model, pooled over severity and expected-QALY gap, using mean paired probability shift pB(2)−pB(1). Each row is one attribute; each colored marker is one model’s point estimate, with horizontal bars showing 95% t-intervals on the run-level paired shift. The vertical reference line marks zero.
τ​(S,G,I,D,X)=𝔼​[PB∣S,G,Z=1,I,D,X]
−𝔼​[PB∣S,G,Z=0,X].(1)
Figure 4: Moralized-Framing Comparison by Model (Pooled over Severity and Expected-QALY Gap). Moralized-framing comparison by model and attribute, pooled over severity and expected-QALY gap, showing the difference (delta) in mean paired probability shift between moralized and neutrally framed scenarios. Horizontal bars show 95% t-intervals. Flip-rate view in Figure 12 (appendix).
Figure 4: Moralized-Framing Comparison by Model (Pooled over Severity and Expected-QALY Gap). Moralized-framing comparison by model and attribute, pooled over severity and expected-QALY gap, showing the difference (delta) in mean paired probability shift between moralized and neutrally framed scenarios. Horizontal bars show 95% t-intervals. Flip-rate view in Figure 12 (appendix).
Figure 5: Case ID placebo controls. Clinically irrelevant case-ID parity information does not produce systematic shifts.
Figure 5: Case ID placebo controls. Clinically irrelevant case-ID parity information does not produce systematic shifts.
(b) Patient A has an odd case ID and Patient B has an even case ID.
(b) Patient A has an odd case ID and Patient B has an even case ID.

Why it matters

As AI systems get folded into sensitive decisions like medical, hiring, or loan allocation, this shows that outcomes can flip based purely on how a question is framed or whether conversation history is kept, not just on the model's underlying judgment. This matters for anyone deploying LLM-based systems, since prompt design and context management turn out to be fairness issues, not just implementation details.

Figure 6: Null placebo controls. Explicitly null and implicit exact-repeat controls do not produce systematic shifts.
Figure 6: Null placebo controls. Explicitly null and implicit exact-repeat controls do not produce systematic shifts.
(b) The new patient information sentence is absent; the original scenario is repeated twice verbatim.
(b) The new patient information sentence is absent; the original scenario is repeated twice verbatim.
Figure 7: Attribute Comparison by Model (Pooled over Severity and Expected-QALY Gap), Flip Rate. Attribute comparison by model, pooled over severity and expected-QALY gap, using flip rate to Person B. A flip occurs when the baseline probability for Person B is at most 0.5 and the updated probability exceeds 0.5. Companion to Figure 3.
Figure 7: Attribute Comparison by Model (Pooled over Severity and Expected-QALY Gap), Flip Rate. Attribute comparison by model, pooled over severity and expected-QALY gap, using flip rate to Person B. A flip occurs when the baseline probability for Person B is at most 0.5 and the updated probability exceeds 0.5. Companion to Figure 3.

Terms in this paper

  • paired-inference · Re-asking the same question while the model can still see its own previous answer in the conversation
  • independent-inference · Asking the same question again with no memory of the prior answer, as a fresh conversation
  • QALY (quality-adjusted life year) · A measure combining survival time and health quality, used to compare treatment benefits
  • flip rate · The rate at which a patient's priority probability crosses from below 0.5 to above 0.5 after new information is added
  • moralized framing · Rewording the same fact using more emotionally or normatively loaded language, e.g. 'disadvantaged' instead of 'low-income'
Figure 8: Severity Comparison by Attribute and Model (Pooled over Expected-QALY Gaps), Absolute Shift. Severity comparison by attribute and model, pooled over expected-QALY gap, using the difference in mean paired probability shift. The minor setting determines who receives treatment first; the severe setting allocates the final remaining treatment slot. Points represent the mean difference between low- and high-severity mean shifts. Horizontal bars represent 95% t-intervals.
Figure 8: Severity Comparison by Attribute and Model (Pooled over Expected-QALY Gaps), Absolute Shift. Severity comparison by attribute and model, pooled over expected-QALY gap, using the difference in mean paired probability shift. The minor setting determines who receives treatment first; the severe setting allocates the final remaining treatment slot. Points represent the mean difference between low- and high-severity mean shifts. Horizontal bars represent 95% t-intervals.
Figure 9: Severity Comparison by Attribute and Model (Pooled over Expected-QALY Gaps), Flip Rate. Severity comparison by attribute and model, pooled over expected-QALY gap, using flip rate to Person B. Values are percentages of paired runs that crossed the 0.5 threshold after the contextual update.
Figure 9: Severity Comparison by Attribute and Model (Pooled over Expected-QALY Gaps), Flip Rate. Severity comparison by attribute and model, pooled over expected-QALY gap, using flip rate to Person B. Values are percentages of paired runs that crossed the 0.5 threshold after the contextual update.
Figure 10: Expected-QALY Gap Comparison by Attribute and Model (Pooled over Severity), Absolute Shift. Expected-QALY gap comparison by attribute and model, pooled over severity, using the difference in mean paired probability shift. The smaller clinical gap generally allows larger movement toward Person B.
Figure 10: Expected-QALY Gap Comparison by Attribute and Model (Pooled over Severity), Absolute Shift. Expected-QALY gap comparison by attribute and model, pooled over severity, using the difference in mean paired probability shift. The smaller clinical gap generally allows larger movement toward Person B.

Original abstract (English)

Large language models are being incorporated into sensitive and important decision-making processes across nearly all fields. While prior work studies model bias around inputs and scenario framing, models can also behave in unexpected and undesirable ways due to context accumulated over their deployment. In this work, we study a medical example in which a model is asked to assign resource-allocation probabilities to two people given brief clinical context, and then sees the same scenario with a single extra sentence containing contrasting patient information, either with or without its previous response in context. Across three of four tested models, the paired-context and independent-inference experiments have different probability shifts, often in opposite directions (in favor of Person B vs. in favor of Person A) when new information is provided. We include additional paired-context experiments to show the effect of varying attributes across scenario axes. Our findings show the context-dependent effect of patient information in a sensitive medical use case. More broadly, our work shows the importance of carefully incorporating LLM-based systems into decision-making processes, context engineering, and further model behavioral studies.

Authors · Spencer Gibson, Tyler Crosse, Magnus Saebo, Achyutha Menon, Eyon Jang, Diogo Cruz

Read on arXiv

Latest papers

All papers →

Latest from METAL MEDIA

Figures: Spencer Gibson et al., arXiv:2608.18108, CC BY 4.0