Everything K-culture — comebacks to K-beauty, straight to your inboxGet it in your inbox

METAL MEDIA

Persona-Guided LLM Agents for Task-Oriented Dialogue

arXiv:2608.180852026-08-20

AI booking assistants get more likeable when they match a user's personality, but start bending the truth more

Researchers tested what happens when an AI assistant that helps book hotels or restaurants adjusts its responses to match a simulated user's personality, such as being outgoing or anxious. They had two AI agents talk to each other under three conditions: the assistant knowing nothing about the user's personality, guessing it from conversation cues, or being told it outright. Personality-aware responses boosted user satisfaction and how well requests were met, but also made the assistant less truthful.

METAL MEDIA explanatory visual

AI booking assistants get more likeable when they match a user's personality, but start bending the truth more

  1. 01The team used three large language models, GPT-4o, Qwen3-Next-80B, and Gemini 2.0 Flash, to simulate a personality-driven 'user' AI talking with a task-completing 'system' AI in hotel and restaurant booking scenarios.
  2. 02Three system conditions were compared: Neutral (no personality info), Try (infer personality from dialogue cues), and Oracle (personality given explicitly).
  3. 03Both personality-aware conditions improved how well user requirements were satisfied, how much useful information was given, and overall user satisfaction, but reduced truthfulness compared to Neutral.
  4. 04When users expressed their assigned trait more strongly, Oracle's satisfaction gains grew accordingly, while Try's gains stayed roughly the same regardless of how clearly the trait showed up, making Try more reliable overall.
  5. 05Introverted and closed-off personalities were much harder for the AI to convincingly express, and Qwen3 and Gemini sometimes mirrored a hostile, disagreeable user's tone instead of handling it constructively, unlike GPT-4o.
An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.

What they did

  1. The team used three large language models, GPT-4o, Qwen3-Next-80B, and Gemini 2.0 Flash, to simulate a personality-driven 'user' AI talking with a task-completing 'system' AI in hotel and restaurant booking scenarios.
  2. Three system conditions were compared: Neutral (no personality info), Try (infer personality from dialogue cues), and Oracle (personality given explicitly).
  3. Both personality-aware conditions improved how well user requirements were satisfied, how much useful information was given, and overall user satisfaction, but reduced truthfulness compared to Neutral.
  4. When users expressed their assigned trait more strongly, Oracle's satisfaction gains grew accordingly, while Try's gains stayed roughly the same regardless of how clearly the trait showed up, making Try more reliable overall.
  5. Introverted and closed-off personalities were much harder for the AI to convincingly express, and Qwen3 and Gemini sometimes mirrored a hostile, disagreeable user's tone instead of handling it constructively, unlike GPT-4o.
Table 1: Trait-level task and personality realization outcomes averaged across domains, models, and the three system personality-access conditions. Dialogue Completion and Task Success summarize schema-guided task performance. User Trait Score measures dialogue-level realization of the intended user trait in generated task-oriented dialogues, while BFI Score measures prompt-level semantic alignment with the intended Big Five direction using a separate questionnaire probe.
TraitTask OutcomesPersonality Realization
Dialogue Completion (%)Task Success (%)User Trait ScoreBFI Score
Extraversion79.6782.124.9965.000
Agreeableness95.8987.444.9004.963
Conscientiousness93.8983.344.3135.000
Neuroticism77.2275.664.9714.967
Openness85.3381.124.9204.967
introversive96.3491.340.6534.625
disagreeable78.7876.224.7735.000
unconscientious96.3495.223.1694.778
stable97.8988.223.4734.917
closed98.1192.000.0274.667
Average89.9485.273.6204.888
Table 2: Trait-level system-quality scores and pairwise user-satisfaction preferences averaged across domains and LLMs. N, T, and O denote Neutral, Try, and Oracle, respectively. CS, IR, and TR denote Constraint Satisfaction, Inform Rate, and Truthfulness, each reported on a 1–5 scale, where higher is better. Pairwise satisfaction cells report win-rate percentages for the first condition over the second; for example, O/N = 62/38 indicates that Oracle is preferred in 62% of comparisons and Neutral in 38%. Values may sum to less than 100 due to ties.
TraitCSIRTRPairwise Satisfaction
NTONTONTOO/NT/NT/O
Extraversion4.3134.3904.3074.5734.6504.5434.6474.2473.88776.00/24.0064.33/35.0040.00/58.66
Agreeableness4.7504.8974.8404.8534.9604.9374.9274.8934.83071.66/27.3466.34/31.0041.33/57.00
Conscientiousness4.6534.7834.7904.7804.9104.9174.9374.8504.88365.34/34.6659.00/39.6746.34/53.33
Neuroticism4.1134.2633.9204.6734.7574.6774.5704.2574.17063.00/36.6769.34/30.6662.66/37.00
Openness4.2134.3534.4474.7634.8034.7834.5474.2233.73385.00/15.0060.33/39.3429.66/70.34
introversive4.7534.9174.8804.8574.9534.9634.9304.8674.89039.33/59.3471.00/26.0073.67/25.33
disagreeable3.7974.0934.0634.2834.5504.5534.6534.5474.65728.33/71.6747.00/53.0080.66/19.34
unconscientious4.7204.9174.8834.8004.9504.9274.8804.8834.85359.00/40.6766.67/32.6665.34/33.66
stable4.8274.9604.9204.9034.9834.9634.9004.9104.87358.34/40.3467.00/30.6653.33/45.00
closed4.8774.8834.9334.9274.9704.9834.9104.9434.97750.33/49.0063.00/34.0068.00/30.34
Average4.5024.6464.5984.7414.8494.8254.7904.6624.57559.63/39.8763.40/35.2056.10/43.00
Table 3: Spearman correlations between Avg. User Trait Score and pairwise satisfaction deltas at two levels of analysis. Trait-level uses one point per trait (n=10), averaged across models and domains. Cell-level uses model × domain × trait cells (n=60, Tables 8–9). p∗<.05; p∗⁣∗∗<.001.
Trait-level (n=10)Cell-level (n=60)
Delta𝝆p𝝆p
ΔO​-​N (Oracle − Neutral)+.685.029∗+.421<.001∗⁣∗∗
ΔT​-​N (Try − Neutral)−.212.556−.157.232
Table 4: Task success rates (%) across models, domains, and personality-access conditions. In the Hotel domain, Neutral achieves the highest cross-model average task success, whereas in the Restaurant domain, Oracle performs best, followed by Try.
Domain / ModelNeutralTryOracle
Hotel
Gemini 2.0 Flash84.4078.0077.40
GPT-4o94.6093.4092.80
Qwen3-Next-80B95.6096.2096.40
Cross-model Average91.5389.2088.87
Restaurant
Gemini 2.0 Flash70.2073.6076.00
GPT-4o80.0085.2088.40
Qwen3-Next-80B84.0084.2084.00
Cross-model Average78.0781.0082.80
Table 5: Dialogue-completion rates (%) for GPT-4o, Qwen3, and Gemini across personality traits in the Hotel and Restaurant domains. N, T, and O denote Neutral, Try, and Oracle, respectively. The average row reports the mean dialogue-completion rate across the ten personality traits for each model and domain.
ModelTraitHotel (%)Restaurant (%)
NTONTO
GPT-4oExtraversion96.090.094.082.098.094.0
Agreeableness100.0100.098.082.096.098.0
Conscientiousness96.094.092.086.096.092.0
Neuroticism84.082.070.080.080.072.0
Openness90.080.090.078.088.090.0
Introversive96.096.096.088.098.098.0
disagreeable84.082.084.056.084.078.0
unconscientious100.0100.0100.074.088.098.0
stable100.0100.0100.094.098.094.0
closed98.0100.098.098.090.096.0
Average94.492.492.281.891.691.0
Qwen3Extraversion70.066.054.080.084.082.0
Agreeableness92.090.092.0100.0100.0100.0
Conscientiousness96.098.098.0100.0100.0100.0
Neuroticism80.084.076.082.088.084.0
Openness88.084.088.090.080.084.0
Introversive86.094.098.0100.0100.0100.0
disagreeable68.074.074.076.078.086.0
unconscientious98.0100.0100.0100.0100.0100.0
stable96.0100.096.0100.0100.0100.0
closed92.098.0100.0100.0100.0100.0
Average86.688.887.692.893.093.6
GeminiExtraversion50.060.052.096.094.092.0
Agreeableness96.096.092.0100.098.096.0
Conscientiousness70.094.090.096.096.096.0
Neuroticism80.072.066.068.066.076.0
Openness80.084.074.094.082.092.0
Introversive98.0100.088.0100.0100.098.0
disagreeable66.070.076.090.092.0100.0
Table 6: Restaurant-domain system-evaluation results for GPT-4o, Qwen3, and Gemini across personality traits under Neutral (N), Try (T), and Oracle (O) conditions. The table reports Constraint Satisfaction, Truthfulness, and Inform Rate; the average row reports the mean across the ten personality traits for each model.
ModelTraitConstraint SatisfactionTruthfulnessInform Rate
NTONTONTO
GPT-4oExtraversion4.224.884.944.904.664.784.385.004.98
Agreeableness4.324.844.904.804.584.644.384.845.00
Conscientiousness4.604.944.764.864.824.824.764.984.84
Neuroticism4.304.404.044.124.023.844.624.684.48
Openness4.264.544.604.644.704.664.444.584.66
Introversive4.484.904.844.984.764.904.624.924.98
disagreeable3.164.124.404.504.284.303.324.564.58
unconscientious4.044.764.864.784.924.704.064.764.96
stable4.845.004.884.764.764.664.805.004.94
closed4.804.684.725.005.004.944.924.884.90
Average4.304.714.694.734.654.624.434.824.83
Qwen3Extraversion4.684.704.624.123.723.664.764.784.70
Agreeableness4.965.005.005.005.004.924.965.005.00
Conscientiousness5.005.004.964.965.004.945.005.004.96
Neuroticism3.924.223.924.744.524.864.684.784.86
Openness4.104.444.524.704.263.404.784.824.76
Introversive5.005.005.004.944.944.965.005.005.00
disagreeable2.783.183.144.384.724.744.164.384.36
unconscientious5.005.005.005.004.985.005.005.005.00
stable4.945.004.985.005.005.005.005.005.00
closed5.005.005.005.005.005.005.005.005.00
Average4.544.654.614.784.714.654.834.884.86
GeminiExtraversion4.924.944.704.944.684.204.964.964.76
Agreeableness5.004.924.884.964.884.985.004.924.90
Conscientiousness4.864.924.965.004.644.864.964.965.00
Neuroticism4.084.144.204.924.704.424.624.784.82
Openness4.764.544.724.844.484.484.904.884.86
Introversive4.985.004.965.005.004.925.005.005.00
disagreeable4.644.844.804.964.804.984.664.905.00
Table 7: Hotel-domain system-evaluation results for GPT-4o, Qwen3, and Gemini across personality traits under Neutral (N), Try (T), and Oracle (O) conditions. The table reports Constraint Satisfaction, Truthfulness, and Inform Rate; the average row reports the mean across the ten personality traits for each model.
ModelTraitConstraint SatisfactionTruthfulnessInform Rate
NTONTONTO
GPT-4oExtraversion4.924.764.744.744.364.144.964.824.80
Agreeableness5.005.005.004.965.004.845.005.005.00
Conscientiousness4.824.684.824.944.924.864.944.844.96
Neuroticism4.424.584.104.624.244.244.844.804.80
Openness4.304.244.584.483.743.364.744.764.88
Introversive4.824.884.925.004.824.824.904.925.00
disagreeable4.684.604.404.824.904.824.864.744.72
unconscientious5.005.004.965.005.004.905.005.005.00
stable4.965.005.004.945.004.945.005.005.00
closed5.004.964.925.005.005.005.005.005.00
Average4.794.774.744.854.704.594.924.894.92
Qwen3Extraversion3.763.583.424.183.662.564.684.584.24
Agreeableness4.484.704.524.844.984.824.945.004.88
Conscientiousness4.664.464.564.944.824.964.964.904.94
Neuroticism3.403.683.224.403.643.264.544.624.42
Openness3.303.704.163.763.822.704.904.864.90
Introversive4.324.724.884.724.724.884.724.884.96
disagreeable3.663.803.524.403.984.284.284.324.34
unconscientious4.644.784.844.784.824.824.964.964.94
stable4.564.764.844.824.824.844.884.964.92
closed4.624.865.004.624.804.984.784.945.00
Average4.144.304.304.554.414.214.764.804.75
GeminiExtraversion3.383.483.425.004.403.983.703.763.78
Agreeableness4.744.924.745.004.924.784.845.004.84
Conscientiousness3.984.704.684.924.904.864.064.784.80
Neuroticism4.564.564.044.624.424.404.744.884.68
Openness4.564.664.104.864.343.804.824.924.64
Introversive4.925.004.684.944.964.864.905.004.84
disagreeable3.864.024.124.864.604.824.424.404.32
Table 8: Hotel-domain pairwise user-satisfaction win rates with averaged personality-realization metrics for GPT-4o, Qwen3, and Gemini. O, T, and N denote Oracle, Try, and Neutral, respectively. Each comparison cell reports the win rate of the first condition over the second condition; for example, O/N = 36/58 means Oracle is preferred in 36% of paired comparisons and Neutral in 58%. Percentages may sum to less than 100 when ties occur. BFI Score is the intended-direction score from the BFI probe for the target trait. Avg. User Trait Score and Avg. Rank are averaged across Oracle, Try, and Neutral cases. Lower Avg. Rank indicates stronger realization of the intended trait.
ModelTraitO / N (%)T / N (%)T / O (%)BFI ScoreAvg. User Trait ScoreAvg. Rank
GPT-4ointroversive36/5864/2670/244.5000.2073.913
closed28/7050/4072/264.1000.0203.987
stable44/5660/3462/364.7503.1732.653
Extraversion80/2064/3634/625.0004.9931.000
Agreeableness76/2056/3644/524.8894.8871.000
Conscientiousness46/5444/5442/565.0004.0471.327
Neuroticism54/4460/4058/405.0004.9201.007
Openness84/1654/4628/725.0004.7931.013
disagreeable44/5638/6266/345.0004.5801.020
unconscientious44/5454/4464/304.4442.9402.187
Average53.6/44.854.4/41.854.0/43.24.7683.4561.911
Qwen3introversive52/4856/4044/564.5000.2673.873
closed60/3856/4444/565.0000.0004.000
stable66/3454/4434/665.0003.5531.833
Extraversion60/4058/4254/465.0005.0001.000
Agreeableness76/2468/3230/705.0004.8201.000
Conscientiousness72/2848/5232/685.0004.0531.053
Neuroticism70/3074/2660/405.0004.9931.000
Openness88/1250/5016/845.0004.9871.000
disagreeable22/7834/6674/265.0004.7801.020
unconscientious60/4068/3054/465.0002.1932.553
Average62.6/37.256.6/42.644.2/55.84.9503.4651.833
Geminiintroversive30/7070/3086/144.8751.4873.133
closed54/4664/3674/264.9000.0603.980
stable48/5076/2464/365.0003.6931.753
Extraversion68/3268/3244/545.0004.9931.000
Agreeableness76/2470/3044/565.0004.9931.000
Conscientiousness64/3666/3464/365.0004.8401.000
Neuroticism70/3084/1678/225.0005.0001.000
Openness92/892/834/664.9004.9801.007
Table 9: Restaurant-domain pairwise user-satisfaction win rates with averaged personality-realization metrics for GPT-4o, Qwen3, and Gemini. O, T, and N denote Oracle, Try, and Neutral, respectively. Each comparison cell reports the win rate of the first condition over the second condition, e.g., O/N = 62/38 means Oracle is preferred in 62% of paired comparisons and Neutral in 38%. Percentages may sum to less than 100 when ties occur. BFI Score is the intended-direction score from the BFI probe for the target trait. Avg. User Trait Score and Avg. Rank are averaged across Oracle, Try, and Neutral cases. Lower Avg. Rank indicates stronger realization of the intended trait.
ModelTraitO / N (%)T / N (%)T / O (%)BFI ScoreAvg. User Trait ScoreAvg. Rank
GPT-4ointroversive62/3882/1860/404.5001.4203.173
closed54/4658/3660/344.1000.3933.760
stable66/3476/2462/364.7503.2602.213
Extraversion88/1266/3228/705.0004.9731.000
Agreeableness76/2478/1838/564.8894.5931.020
Conscientiousness70/3070/3054/465.0003.5071.807
Neuroticism50/5058/4262/385.0004.6731.020
Openness80/2060/3840/605.0004.8271.020
disagreeable70/3064/3664/365.0004.5271.000
unconscientious70/3078/2268/324.4441.2603.413
Average68.6/31.469.0/29.653.6/44.84.7683.3431.943
Qwen3introversive4/9670/2698/24.5000.2403.867
closed42/5872/2684/165.0000.0004.000
stable76/2270/2638/605.0003.5801.747
Extraversion72/2862/3648/525.0004.9871.000
Agreeableness88/1068/2818/825.0004.7871.013
Conscientiousness74/2676/1838/625.0004.0731.020
Neuroticism50/5068/3270/305.0004.9801.000
Openness82/1852/4820/805.0004.9131.040
disagreeable20/8058/4286/145.0004.9601.000
unconscientious60/4070/3072/285.0002.0272.660
Average56.8/42.866.6/31.257.2/42.64.9503.4551.835
Geminiintroversive52/4684/1684/164.8751.8952.969
closed64/3678/2274/244.9000.1403.940
stable50/4666/3260/365.0003.6601.820
Extraversion88/1268/3232/685.0005.0001.000
Agreeableness38/6258/4274/265.0005.0001.000
Conscientiousness66/3450/5048/525.0004.6001.000
Neuroticism84/1672/2848/525.0005.0001.000
Openness84/1654/4640/604.9004.9531.087
Table 10: Per-trait Avg. Intended Score and average pairwise satisfaction deltas (pp), averaged across six model–domain combinations. Traits are sorted by Avg. Intended Score in descending order. The disagreeable trait is discussed separately in Appendix C.3.
TraitAvg. Intended ScoreAvg. ΔO−N (%)Avg. ΔT−N (%)
Extraversion4.991+52.0+29.3
Neuroticism4.928+26.3+38.7
Openness4.909+70.0+21.0
Agreeableness4.847+44.3+35.3
disagreeable†4.788−43.3−6.0
Conscientiousness4.187+30.7+19.3
stable3.486+18.0+36.3
unconscientious2.737+18.3+34.0
introversive0.919−20.0+45.0
closed0.102+1.3+29.0

Why it matters

This shows a concrete tradeoff for anyone building customer-facing chat assistants: tailoring tone to a user's personality can raise satisfaction but risks the assistant fabricating or distorting information. It also demonstrates that this kind of personality-aware behavior can be achieved through prompting alone, without retraining the underlying model.

Terms in this paper

  • Big Five traits · a psychology framework describing personality across five dimensions: extraversion, agreeableness, conscientiousness, neuroticism, and openness
  • task-oriented dialogue (TOD) · multi-turn conversation aimed at completing a concrete goal, like booking a table
  • Constraint Satisfaction · a metric checking whether the final recommendation or booking actually meets the user's stated requirements
  • Truthfulness · a metric checking whether the system avoids making up unsupported claims and stays grounded in real dialogue and data
  • Spearman correlation · a rank-based statistic measuring whether two variables tend to rise and fall together

Figures we cannot republish

  • Figure 1: Overview of our framework for personality-aware task-oriented dialogue. A personality-conditioned user agent interacts with a system agent under three personality-access conditions—Neutral, Try, and Oracle—within an SGD-based API-grounded task setting. Generated dialogues are evaluated for task outcomes, system quality, and personality-related behavior.
  • Figure 2: Spearman correlation between Avg. User Intended Score and pairwise satisfaction deltas across traits (n=10). Oracle gains increase with trait realization (ρ=+0.685, p=.029), whereas Try gains do not (ρ=−0.103, p=.777). Point size indicates cross-dataset SD; disagreeable reflects a Qwen3/Gemini mirroring artifact (Appendix C.3).
  • Figure 3: Representative GPT-4o Restaurant-domain score-delta figures. The top panel shows score changes from Neutral to Try, and the bottom panel shows score changes from Neutral to Oracle.
See the figures in the original paper →

Original abstract (English)

Prior work has shown that large language models (LLMs) can express diverse personality traits in open-ended text generation. However, it remains unclear whether they can do so in a goal-directed dialogue without compromising task completion, and whether adapting to the user's personality improves the interaction quality. We study these questions in task-oriented dialogue (TOD), where a system helps a user accomplish a goal via multi-turn interaction. We build a training-free framework that simulates a TOD interaction between two LLMs: a user agent that exhibits a target personality and a system agent that adapts to the user while completing the task. To isolate the effect of adaptation, we vary how much the system knows about the user's personality across three conditions. In Neutral, the system receives no personality information. In Try, it infers the personality from dialogue cues. In Oracle, it is given the personality explicitly. We evaluate GPT-4o, Qwen3-Next-80B, and Gemini 2.0 Flash on Hotel and Restaurant dialogues from the Schema-Guided Dialogue (SGD) dataset, across the Big Five traits and their opposite poles. We find that the user agent can express personality while the system maintains strong task performance, although some traits are realized far less reliably than others. Adapting to the user's personality improves constraint satisfaction, inform rate, and user satisfaction, but lowers truthfulness, revealing a trade-off between personalization and task-grounding. Oracle's gains grow when the target trait is strongly expressed, whereas Try's gains are largely insensitive to realization strength. Overall, cue-based adaptation in Try best resolves this trade-off and offers a more reliable route to personality-aware TOD without fine-tuning.

Authors · Maryam Shoaeinaeini, Brent Harrison, A. B. Siddique

Read on arXiv

Latest papers

All papers →

Latest from METAL MEDIA