Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents

arXiv:2608.150082026-08-14

把AI智能体的11种记忆存储方式放在同一条件下比拼,发现没有一种能通吃所有场景

研究者对长时间运行的大模型智能体常用的11种记忆存储方式做了统一对照评测,涵盖向量检索、知识图谱、层级摘要、参数微调等七大类。结果显示知识图谱式记忆在对话问答上表现最好,却在具身行动规划任务上拖后腿;而把经验提炼压缩的记忆方式则正好相反。结论是智能体记忆系统需要根据任务动态切换存储方式,而不是固定用一种。

METAL MEDIA 解读图

把AI智能体的11种记忆存储方式放在同一条件下比拼,发现没有一种能通吃所有场景

01对52个现有记忆增强系统的调查发现,62%的评测集中在LoCoMo和LongMemEval两个对话类基准上,只有21%的系统报告了延迟、成本等效率指标
02研究在3个模型骨干、4个基准、26项指标下,统一对比了7大类共11种记忆方式,包括稠密/稀疏向量索引、文本记录、知识图谱、层级树、经验蒸馏记忆、LoRA参数微调、KV缓存压缩等
03在长对话问答任务上,图谱加向量的混合方式(M5)准确率最高,但速度慢10到100倍;在具身行动规划任务(ALFWorld)上,把经验提炼成精简策略的记忆方式获胜,而单纯检索更多原始记忆反而降低任务成功率
04调节检索条数k发现:在长对话问答中准确率随k增大持续上升,但在行动规划任务中任务成功率随k增大而下降,原因是模型的注意力被检索内容抢走,忽略了当前需要关注的场景
05把对话长度从6千个token压力测试到26万个token时,知识图谱类记忆的处理速度大幅下降,而经验蒸馏类记忆在速度和质量上都保持稳定扩展

这是 METAL MEDIA 制作的解读图，并非论文作者提供的原图。

他们做了什么

对52个现有记忆增强系统的调查发现,62%的评测集中在LoCoMo和LongMemEval两个对话类基准上,只有21%的系统报告了延迟、成本等效率指标
研究在3个模型骨干、4个基准、26项指标下,统一对比了7大类共11种记忆方式,包括稠密/稀疏向量索引、文本记录、知识图谱、层级树、经验蒸馏记忆、LoRA参数微调、KV缓存压缩等
在长对话问答任务上,图谱加向量的混合方式(M5)准确率最高,但速度慢10到100倍;在具身行动规划任务(ALFWorld)上,把经验提炼成精简策略的记忆方式获胜,而单纯检索更多原始记忆反而降低任务成功率
调节检索条数k发现:在长对话问答中准确率随k增大持续上升,但在行动规划任务中任务成功率随k增大而下降,原因是模型的注意力被检索内容抢走,忽略了当前需要关注的场景
把对话长度从6千个token压力测试到26万个token时,知识图谱类记忆的处理速度大幅下降,而经验蒸馏类记忆在速度和质量上都保持稳定扩展

Table 1: Memory substrates on LoCoMo, LME-S, and MAB (LRU, TTL, CR). P4 ↑: LLM-judge score; E15 ↓: per-query latency (s). Within each model band, green and red mark the per-column best and worst; bold marks the per-model P4 winner on each benchmark.
			LoCoMo	LME-S	MAB LRU	MAB TTL	MAB CR
Model	Sub.	Family	P4 ↑	𝑬𝟏𝟓 ↓	P4 ↑	𝑬𝟏𝟓 ↓	P4 ↑	𝑬𝟏𝟓 ↓	P4 ↑	𝑬𝟏𝟓 ↓	P4 ↑	𝑬𝟏𝟓 ↓
Qwen3-8B	External
M1	Flat	0.540	0.77	0.478	7.30	0.578	9.94	0.560	0.92	0.270	1.65
M2	Flat	0.470	0.33	0.480	1.75	0.620	3.23	0.470	0.33	0.270	1.39
M3	Text	0.562	4.18	0.495	55.58	0.563	146.65	0.640	27.15	0.240	2.30
M4	Struct	0.435	6.36	0.430	225.67	0.493	274.06	0.480	12.65	0.270	1.50
M5	Struct	0.648	27.84	0.537	187.21	0.578	313.18	0.540	23.70	0.440	6.01
M6	Hier	0.556	1.39	0.507	19.96	0.493	59.98	0.290	3.24	0.290	3.30
M8	Refine	0.509	3.03	0.490	102.85	0.592	197.09	0.560	1.02	0.340	2.07
Internal
M9	Weight	0.379	1.76	0.250	51.50	0.300	85.00	0.220	8.50	0.260	1.95
M10	Act	0.589	4.55	0.120	6.61	0.676	6.58	0.480	2.28	0.270	1.64
M11	Act	0.307	2.03	0.090	14.61	0.634	21.30	0.560	0.56	0.240	1.76
Qwen3-32B-AWQ	External
M1	Flat	0.582	2.59	0.548	12.00	0.676	32.39	0.810	1.81	0.460	4.33
M2	Flat	0.487	2.37	0.497	7.31	0.662	23.14	0.780	1.20	0.410	4.03
M3	Text	0.586	6.47	0.560	67.42	0.775	477.97	0.850	94.29	0.340	3.94
M4	Struct	0.452	8.70	0.490	234.00	0.535	278.93	0.710	17.18	0.190	1.73
M5	Struct	0.683	29.22	0.627	194.08	0.578	333.95	0.630	26.02	0.380	9.38
M6	Hier	0.573	5.20	0.593	20.27	0.577	73.16	0.330	3.77	0.410	8.70
M8	Refine	0.533	5.53	0.533	106.78	0.592	228.66	0.810	1.93	0.440	3.39
Internal
M9	Weight	0.399	4.73	0.290	171.70	0.422	253.63	0.250	22.15	0.280	5.50
M10	Act	0.669	16.25	0.190	22.98	0.620	27.43	0.760	2.43	0.380	3.74
M11	Act	0.274	4.17	0.113	52.39	0.620	25.00	0.810	1.45	0.300	2.65
Gemma-4-26B-A4B-IT	External
M1	Flat	0.574	0.93	0.515	8.49	0.676	16.50	0.770	14.50	0.460	4.09
M2	Flat	0.555	0.49	0.513	3.37	0.662	9.18	0.730	0.20	0.460	3.88
M3	Text	0.599	4.81	0.530	31.78	0.720	238.78	0.750	36.21	0.430	3.67
M4	Struct	0.476	6.61	0.523	230.39	0.535	267.00	0.720	16.12	0.460	3.97

Table 2: Memory substrates evaluated on ALFWorld-unseen and BigCodeBench-Hard. Within each model band, green marks the best value on a metric, red marks the worst. Pavg is the mean per-task partial-success score. Bold marks the per-model performance winner (TSR for ALFWorld, Pass@1 for BCB). M10 is omitted from both agent-centric benchmarks because the cumulative context exceeds the tested context windows; M9 is omitted from the Gemma-4 band because it is incompatible with the A4B MoE routing.
			ALFWorld-unseen	BigCodeBench-Hard
Model	Sub.	Family	TSR ↑	𝐒𝐭𝐞𝐩𝐬\|𝓢 ↓	𝑷𝐚𝐯𝐠 ↑	𝑬𝟐𝐩𝟗𝟎 ↓	𝑬𝟏𝟓 ↓	Pass@1 ↑	𝑷𝐚𝐯𝐠 ↑	𝑬𝟐𝐚𝐯𝐠 ↓	𝑬𝟏𝟓 ↓
Qwen3-8B	-	NoMem	5.7	12.2	13.5	13.2	488	8.1	12.4	3.8	4
M1	Flat	5.2	5.9	6.6	16.1	672	9.5	20.1	5.0	5
M2	Flat	6.7	6.0	9.4	15.7	647	10.1	11.9	4.9	5
M3	Text	5.2	8.3	8.4	16.3	653	8.1	12.4	5.3	5
M4	Struct	7.5	7.9	8.5	14.8	605	10.1	20.4	4.6	5
M5	Struct	9.0	17.4	12.5	15.7	620	15.5	17.1	28.3	28
M6	Hier	4.5	15.0	8.3	15.6	656	12.2	22.2	4.8	7
M7	Refine	7.5	13.7	9.6	16.0	629	12.2	16.9	4.6	17
M8	Refine	8.2	18.5	9.5	15.3	620	14.9	24.0	5.1	19
M9	Weight	3.0	19.5	3.2	22.0	921	7.4	4.6	12.7	13
M11	Act	11.9	14.2	14.5	13.5	553	13.5	23.6	16.8	17
Qwen3-32B-AWQ	-	NoMem	22.4	11.3	23.7	20.3	583	17.6	25.6	3.9	4
M1	Flat	27.6	9.6	16.2	31.1	1.08k	18.2	24.2	5.0	5
M2	Flat	21.6	10.8	15.4	30.8	1.09k	19.6	21.8	5.0	5
M3	Text	26.9	9.9	21.8	30.3	950	18.2	24.5	5.4	8
M4	Struct	26.9	12.4	24.3	28.6	816	17.6	22.4	4.9	20
M5	Struct	23.1	11.4	23.0	27.4	851	16.2	16.6	31.2	31
M6	Hier	23.9	9.9	22.8	26.8	817	16.2	16.8	4.4	7
M7	Refine	32.1	11.1	29.9	27.2	716	17.6	13.5	4.3	21
M8	Refine	22.4	12.4	21.3	28.0	886	13.5	23.0	4.6	22
M9	Weight	29.9	12.1	27.4	21.4	612	14.2	15.5	14.6	15
M11	Act	26.9	12.3	24.7	22.0	783	13.5	13.8	21.3	21
Gemma-4-26B-A4B-IT	-	NoMem	7.5	11.3	17.3	16.4	671	14.2	19.5	19.8	20
M1	Flat	10.4	15.0	16.3	21.6	786	18.2	26.4	21.9	22
M2	Flat	9.7	11.5	17.2	21.4	753	19.6	27.0	20.6	21
M3	Text	10.4	14.8	14.9	20.5	781	14.2	19.5	23.4	23
M4	Struct	8.2	14.5	13.9	22.5	942	18.9	26.9	22.4	23
M5	Struct	4.5	8.5	9.4	19.9	749	20.9	28.4	45.6	46
M6	Hier	10.4	11.4	18.1	21.1	741	18.9	26.0	24.3	28

Table 3: Landscape of 52 memory-augmented LLM systems (2023–2026). Benchmark codes: LC = LoCoMo, LME = LongMemEval, MH = Multi-hop QA, DS = DialSim, DMR = DMR. Metric codes: A = accuracy, P = per-type, T = token, L = latency, R = runtime, M = memory size, K = API calls. #B: distinct benchmarks used. #E: efficiency dimensions reported (bold ≥2). Base: G = GPT-series, O = open-source. Summary: 81% GPT-family; 62% of benchmark pairs on LC + LME; only 21% report any efficiency metric; no system reaches ≥3 benchmarks × ≥3 efficiency dimensions.
System	Year	Bench.	Metrics	#B	#E	Base	System	Year	Bench.	Metrics	#B	#E	Base
Generative Agents [39]	2023	Other	A	1	0	G	Mem0 [3]	2025	LC	A, P, T, L	1	2	G
MemGPT [36]	2023	DMR	A	1	0	G	Mem-α [64]	2025	Other	A, P, M	1	1	O
MemoChat [29]	2023	Other	A	1	0	G	MemAgent [75]	2025	Other	A	1	0	O
MemoryBank [82]	2023	Other	A	1	0	G	MemOS [27]	2025	LC, LME	A, P	2	0	G
RSum [59]	2023	Other	A	1	0	G	Memoria [44]	2025	LME	A, T, L	1	2	G
SCM [55]	2023	Other	A	1	0	G	MemoryOS [19]	2025	LC	A, P	1	0	G
AI PERSONA [61]	2024	Other	A	1	0	G	Memory-R1 [72]	2025	LC, LME, Other	A, P	3	0	O
EM-LLM [7]	2024	Other	A, P	1	0	O	Memory-T1 [5]	2025	LC, Other	A, P	2	0	O
HippoRAG [9]	2024	MH	A, P	1	0	O	MIRIX [63]	2025	LC, Other	A, P, M	2	1	G
LD-Agent [24]	2024	Other	A, P	1	0	G	MMS [77]	2025	LC	A, P	1	0	G
MemTree [42]	2024	Other	A	1	0	G	Nemori [32]	2025	LC, LME	A, P	2	0	G
RAPTOR [45]	2024	Other	A	1	0	G	O-Mem [58]	2025	LC, LME	A, P	2	0	G
ReadAgent [23]	2024	Other	A	1	0	G	PRINCIPLES [21]	2025	Other	A	1	0	G
THEANINE [34]	2024	Other	A	1	0	G	PropRAG [56]	2025	MH, Other	A, P	2	0	O
A-Mem [70]	2025	LC, DS	A, P, T	2	1	G	R3Mem [62]	2025	Other	A	1	0	O
ComoRAG [57]	2025	Other	A	1	0	G	RGMem [53]	2025	LC, Other	A, P	2	0	G
GAM [71]	2025	LC	A	1	0	G	RMM [51]	2025	LC	A, P	1	0	G
H-Mem [74]	2025	LC	A, P, R	1	1	O	SeCom [38]	2025	LC	A, P	1	0	G
Hindsight [22]	2025	LC, LME	A, P	2	0	G	SGMem [66]	2025	LC, LME	A, P	2	0	G
HippoRAG 2 [10]	2025	MH	A, P	1	0	O	EMem [84]	2025	LC, LME	A, P	2	0	G
LightMem [6]	2025	LC, LME	A, P, T, L, R, K	2	4	G	Zep [40]	2025	DMR, LME	A, P, L	2	1	G
LiCoMemory [16]	2025	LC, LME	A, P, L	2	1	G
Agentic Memory [76]	2026	LC	A, P	1	0	G	MemMachine [60]	2026	LC, LME, MH	A, P, T	3	1	G
EverMemOS [11]	2026	LC, LME	A, P	2	0	G	Memori [1]	2026	LC	A, P	1	0	G
Cognis [4]	2026	LC, LME	A, P	2	0	G	MMM [13]	2026	LC, LME	A, P	2	0	G
MAGMA [17]	2026	LC, LME	A, P	2	0	G	SwiftMem [52]	2026	LC, LME	A, P, L	2	1	G
							TSM [48]	2026	LC, LME	A, P	2	0	G

Table 4: Memory substrate configurations. TF: training-free (✓) or requires weight modification (✗). LLM: auxiliary LLM required (✓) beyond final-answer generation. ‡ attention-guided utterance selection with re-prefill (Qwen3 hybrid-attention adaptation). § zero-shot LLM controller in place of PPO.
Family	Method	TF	LLM	Write	Read	Mgmt
External Memory
Flat Index	M1 Dense Vector [68]	✓	✗	Encode; add to index	Dense top-k	—
	M2 Sparse Vector [43]	✓	✗	Tokenize; append to BM25	Sparse top-k	—
Text Record	M3 Gist Index [23]	✓	✓	LLM gist per page; index gists	LLM selects pages to expand	Text de-dup
Structural	M4 Evolving Notes [70]	✓	✓	LLM note per write; build links	ANN + link expansion	Evolution-triggered neighbour rewrites
	M5 Dual-Level Graph [8]	✓	✓	LLM extracts entities + relations; build KG	KG + vector hybrid (mix mode)	Lazy batch indexing; cached graph
Hierarchical	M6 Hierarchical Tree [45, 49]	✓	✓	LLM assigns four levels	Collapsed-tree top-k; all levels	Top-level de-dup; decay
Refinement	M7 Distilled Strategies [35]	✓	✓	LLM judges; distills strategies; MaTTS	Dense over strategies; top-1	Trajectory judge; strategy de-dup
	M8 Skill Bundles§ [79]	✓	✓	Cluster experiences into bundles	Zero-shot LLM; all bundles prefilled	Merge and prune on overflow
Internal Memory
Weight	M9 Adapter Tuning [80]	✗	✓	LLM QA pairs; LoRA answer-only loss	Direct generation on base + adapter	—
Activation	M10 Full Context	✓	✗	Append to buffer	Concatenate; generate	Truncate at window
	M11 Episode-Clustered Re-prefill‡ [20]	✓	✗	Cluster into episodes; keep 25%	Match to centroid; re-prefill episode	Episodic clustering; budgeting

Table 5: Auxiliary-LLM cost on a full LoCoMo run (ten conversations, 1,986 queries; GPT-4o-mini aux model, embedding excluded). Costs are one to two orders of magnitude above the lightest retrieval baselines (e.g. M2 BM25 issues zero aux-LLM calls and finishes in roughly ten minutes).
Substrate	Method	Auxiliary LLM calls	Execution time
Text Record	MemGPT [36]	2,739	16.3 h
Text Record	Mem0 [3]	8,984	11.8 h
Structural	Zep [40]	7,624	23.1 h

Table 6: Metric taxonomy for evaluating memory substrates. Two families: Performance (P1–P6, P8–P12) captures answer accuracy and retrieval quality; Efficiency (E1–E15) captures latency, storage, LLM call costs, and aggregate totals. The Benchmark column records which benchmarks officially report each metric. Abbreviations: LC = LoCoMo, MA = MemoryAgentBench, ALF = ALFWorld, BC = BigCodeBench-Hard; All denotes substrate-level metrics applicable across all four. Arrows indicate the preferred direction.
ID	Metric	Definition	Benchmark
Performance: answer accuracy and retrieval quality
P1 ↑	Exact Match	𝟙[𝚗𝚘𝚛𝚖(y^)=𝚗𝚘𝚛𝚖(y)]; lowercase, strip articles, punctuation, whitespace	LC
P2 ↑	Token F1	Harmonic mean of token level precision and recall between 𝚗𝚘𝚛𝚖⁡(y^) and 𝚗𝚘𝚛𝚖⁡(y)	LC
P3 ↑	BLEU 1	Clipped unigram precision with brevity penalty: BP⋅∑wmin⁡(cwy^,cwy)/\|y^\|	LC
P4 ↑	LLM Judge	LLM labels the answer CORRECT/WRONG given (q,y,y^); fraction correct in [0,1]	LC, MA
P5	Compression Ratio	\|input tokens\|/\|stored tokens\|; larger values indicate more lossy compression (unsigned)	All
P6 ↑	Recall@k	\|𝒢∩ℛ\|/\|𝒢\|; coverage of gold evidence dia_ids	LC, MA
P8 ↑	Task Success	Fraction of episodes where all goal conditions are satisfied (binary per episode); ALFWorld primary metric	ALF
P9 ↑	GC Success	Fraction of individual sub goal conditions satisfied (ALFRED inherited); reported in parentheses alongside P8	ALF
P10 ↓	Steps to Goal	Environment actions per successful episode; auxiliary efficiency signal, not officially reported by ALFWorld	ALF
P11 ↑	Pass@1	Unit test pass rate under greedy decoding; BigCodeBench primary metric	BC
P12 ↑	SubEM	𝟙[𝚗𝚘𝚛𝚖(y)⊆𝚗𝚘𝚛𝚖(y^)]; substring exact match. MA primary metric for Accurate Retrieval and Conflict Resolution	MA
Efficiency: latency, storage, LLM call costs, and aggregate totals
E1 ↓	Memory Size	Memory unit’s data structures in bytes; for parametric methods (M9), includes edited weights or adapter size	All
E2 ↓	Inference Time	Total per query wall time: read() + generate(); mean, p50, p90 (ms)	All
E3 ↓	Write Latency	Per message write() wall time; mean, p50, p90 (ms)	All
E4 ↓	Retrieval Latency	Per query read() wall time; mean, p50, p90 (ms)	All
E5 ↓	Write Tokens	LLM tokens consumed during the write phase (input + output)	All
E6 ↓	Write Calls	Number of LLM calls during the write phase	All
E7	Retrieved Tokens	Tokens surfaced by memory to support answer generation	All
E8 ↓	Read Tokens	LLM tokens consumed during read phase calls (input + output)	All
E9 ↓	Read Calls	Number of LLM calls during the read phase	All
E10 ↓	Mgmt. Tokens	LLM tokens for memory management (dedup, update, delete, conflict resolution)	All
E11 ↓	Mgmt. Calls	Number of LLM calls for memory management	All
E12 ↓	Total Tokens	Aggregate auxiliary-LLM tokens across phases: E5+E8+E10 (excludes retrieved tokens E7)	All
E13 ↓	Total Calls	Aggregate LLM calls across phases: E6+E9+E11	All
E14 ↓	Total Wall-clock	End-to-end run time across the benchmark (s); includes write, read, and management phases	All
E15 ↓	Per-query Latency	E14/nq, the principal cost reported in the main tables (s/query)	All

Table 7: Table for different substrate and model LoCoMo Performance.
Model	Sub.	Family	P1 EM	P2 F1	P3 BLEU	P4 Judge	P6 Recall@k
Qwen3-8B	M1	Flat	0.273	0.466	0.285	0.540	0.696
M2	Flat	0.283	0.436	0.241	0.470	0.595
M3	Text	0.190	0.366	0.269	0.562	0.620
M4	Struct	0.226	0.355	0.216	0.435	0.465
M5	Struct	0.196	0.381	0.252	0.648	0.455
M6	Hier	0.251	0.436	0.276	0.556	0.797
M8	Refine	0.301	0.456	0.252	0.509	0.615
M9	Weight	0.019	0.099	0.071	0.379	–
M10	Act	0.159	0.342	0.285	0.589	–
M11	Act	0.066	0.161	0.138	0.307	–
Qwen3-32B-AWQ	M1	Flat	0.208	0.408	0.328	0.582	0.696
M2	Flat	0.187	0.352	0.272	0.487	0.595
M3	Text	0.173	0.368	0.289	0.586	0.620
M4	Struct	0.165	0.320	0.240	0.452	0.465
M5	Struct	0.134	0.329	0.286	0.683	0.455
M6	Hier	0.192	0.387	0.307	0.573	0.797
M8	Refine	0.190	0.365	0.281	0.533	0.615
M9	Weight	0.024	0.101	0.082	0.399	–
M10	Act	0.190	0.412	0.337	0.669	–
M11	Act	0.035	0.118	0.095	0.274	–
Gemma-4-26B	M1	Flat	0.329	0.513	0.311	0.574	0.696
M2	Flat	0.302	0.451	0.257	0.555	0.595
M3	Text	0.303	0.484	0.300	0.599	0.620
M4	Struct	0.310	0.405	0.232	0.476	0.465
M5	Struct	0.203	0.451	0.271	0.719	0.455
M6	Hier	0.310	0.483	0.292	0.592	0.797
M8	Refine	0.335	0.482	0.241	0.554	0.615
M10	Act	0.300	0.515	0.308	0.688	–
M11	Act	0.085	0.181	0.149	0.330	–

Table 8: Table for different substrate and model LoCoMo Latency & Quality.
Model	Sub.	Family	𝑬𝟐 (ms/q)	𝑬𝟑 (ms/wr)	𝑬𝟒 (ms/rd)	𝑬𝟏𝟒 (s)	𝑬𝟏𝟓 (s/q)	𝑷𝟓
Qwen3-8B	M1	Flat	522	199	226	1,534	0.77	1.00
M2	Flat	302	0	1	660	0.33	1.00
M3	Text	2,292	1,663	781	8,293	4.18	5.29
M4	Struct	169	6,210	9	12,638	6.36	0.71
M5	Struct	20,680	5,116	9,610	55,290	27.84	0.44
M6	Hier	683	641	195	2,758	1.39	0.62
M8	Refine	431	2,570	179	6,019	3.03	1.09
M9	Weight	335	1,400	–	3,498	1.76	–
M10	Act	4,132	–	–	9,027	4.55	–
M11	Act	1,848	–	–	4,037	2.03	–
Qwen3-32B-AWQ	M1	Flat	2,174	194	648	5,142	2.59	1.00
M2	Flat	2,153	0	1	4,703	2.37	1.00
M3	Text	4,379	1,623	1,100	12,852	6.47	5.19
M4	Struct	2,289	6,358	12	17,269	8.70	0.72
M5	Struct	21,949	5,221	9,586	58,025	29.22	0.43
M6	Hier	2,650	653	610	10,327	5.20	0.60
M8	Refine	2,707	2,600	166	10,991	5.53	1.11
M9	Weight	860	3,800	–	9,386	4.73	–
M10	Act	14,774	–	–	32,275	16.25	–
M11	Act	3,790	–	–	8,280	4.17	–
Gemma-4-26B	M1	Flat	667	200	188	1,850	0.93	1.00
M2	Flat	445	0	1	972	0.49	1.00
M3	Text	2,870	1,664	1,018	9,555	4.81	5.37
M4	Struct	394	6,145	11	13,129	6.61	0.73
M5	Struct	18,982	5,135	9,444	51,544	25.95	0.43
M6	Hier	712	626	174	2,822	1.42	0.61
M8	Refine	600	2,572	166	6,388	3.22	1.06
M10	Act	4,698	–	–	10,263	5.17	–
M11	Act	1,967	–	–	4,297	2.16	–

Table 9: Table for different substrate LoCoMo Storage, Tokens & Calls.
Sub.	Family	𝑬𝟏	𝑬𝟓	𝑬𝟔	𝑬𝟕	𝑬𝟖	𝑬𝟗	𝑬𝟏𝟎	𝑬𝟏𝟏	𝑬𝟏𝟐	𝑬𝟏𝟑
M1	Flat	3.59M	0	0	889.8K	0	0	0	0	0	0
M2	Flat	173.7K	0	0	806.6K	0	0	0	0	0	0
M3	Text	257.4K	311.8K	390	11.52M	8.23M	2,040	0	0	8.54M	2,430
M4	Struct	177.6K	4.59M	3,268	96.2K	0	0	3.16M	2,312	7.75M	5,580
M5	Struct	167.8K	33.0M	9,860	6.00M	17.8M	3,980	0	0	50.8M	13,840
M6	Hier	4.33M	381.8K	910	1.87M	0	0	127.8K	190	509.6K	1,100
M8	Refine	3.40M	1.46M	481	806.2K	2.94M	2,037	242.9K	51	4.64M	2,569
M9	Weight	host-dep	–	–	–	–	–	–	–	–	–
M10	Act	101.2K	–	–	60.9M	–	–	–	–	–	–
M11	Act	100.7K	–	–	15.3M	–	–	–	–	–	–

Table 10: P12 (SubEM), P2 (F1), P3 (BLEU-1) per (substrate, generator, sweep) for LongMemEval-S and MemoryAgentBench.
			LME-S	MAB LRU	MAB TTL	MAB CR
Model	ID	Family	P12	P2	P3	P12	P2	P3	P12	P2	P3	P12	P2	P3
Qwen3-8B	M1	Flat	0.390	0.170	0.113	0.592	0.491	0.417	0.570	0.476	0.405	0.330	0.211	0.171
M2	Flat	0.340	0.170	0.114	0.606	0.527	0.448	0.480	0.399	0.340	0.280	0.188	0.155
M3	Text	0.123	0.047	0.029	0.549	0.479	0.407	0.840	0.544	0.462	0.270	0.192	0.163
M4	Struct	0.303	0.145	0.094	0.465	0.419	0.356	0.490	0.468	0.397	0.290	0.121	0.103
M5	Struct	0.380	0.160	0.103	0.549	0.491	0.417	0.580	0.468	0.397	0.500	0.362	0.308
M6	Hier	0.357	0.162	0.111	0.521	0.419	0.356	0.290	0.246	0.210	0.270	0.171	0.138
M8	Refine	0.373	0.159	0.101	0.592	0.503	0.427	0.960	0.570	0.484	0.310	0.272	0.231
M9	Weight	0.083	0.082	0.058	0.239	0.192	0.163	0.170	0.167	0.142	0.160	0.108	0.089
M10	Act	0.090	0.075	0.053	0.648	0.575	0.488	0.140	0.130	0.111	0.270	0.161	0.124
M11	Act	0.202	0.111	0.065	0.592	0.539	0.458	0.560	0.468	0.397	0.050	0.037	0.032
Qwen3-32B-AWQ	M1	Flat	0.443	0.180	0.122	0.634	0.575	0.488	0.820	0.689	0.585	0.280	0.134	0.093
M2	Flat	0.377	0.163	0.109	0.620	0.563	0.478	0.790	0.663	0.564	0.270	0.136	0.091
M3	Text	0.225	0.124	0.072	0.704	0.658	0.560	0.860	0.722	0.614	0.280	0.158	0.120
M4	Struct	0.357	0.160	0.106	0.521	0.455	0.387	0.680	0.666	0.566	0.140	0.072	0.049
M5	Struct	0.460	0.159	0.104	0.592	0.491	0.417	0.620	0.510	0.433	0.440	0.191	0.127
M6	Hier	0.413	0.176	0.117	0.577	0.623	0.529	0.340	0.281	0.238	0.280	0.135	0.090
M8	Refine	0.403	0.166	0.111	0.577	0.503	0.427	0.940	0.790	0.672	0.310	0.162	0.115
M9	Weight	0.100	0.098	0.065	0.408	0.359	0.305	0.324	0.318	0.270	0.160	0.086	0.059
M10	Act	0.137	0.087	0.061	0.606	0.527	0.448	0.760	0.646	0.549	0.270	0.128	0.085
M11	Act	0.080	0.080	0.056	0.592	0.515	0.438	0.820	0.689	0.585	0.230	0.122	0.086
Gemma-4-26B	M1	Flat	0.433	0.200	0.135	0.662	0.575	0.488	0.770	0.629	0.535	0.520	0.267	0.223
M2	Flat	0.357	0.180	0.122	0.662	0.673	0.572	0.730	0.620	0.527	0.520	0.248	0.202
M3	Text	0.223	0.123	0.071	0.352	0.275	0.234	0.798	0.616	0.524	0.520	0.304	0.259
M4	Struct	0.353	0.166	0.111	0.521	0.455	0.387	0.690	0.616	0.524	0.080	0.068	0.058
M5	Struct	0.453	0.192	0.129	0.535	0.383	0.326	0.720	0.612	0.520	0.660	0.294	0.214
M6	Hier	0.397	0.175	0.118	0.493	0.431	0.366	0.250	0.212	0.181	0.480	0.228	0.182
M8	Refine	0.373	0.164	0.108	0.549	0.455	0.387	0.950	0.807	0.686	0.470	0.327	0.278
M10	Act	0.063	0.045	0.039	0.577	0.579	0.492	0.460	0.391	0.332	0.500	0.263	0.222
M11	Act	0.234	0.129	0.075	0.572	0.443	0.376	0.702	0.616	0.524	0.050	0.050	0.043

Table 11: LongMemEval-S storage and latency per substrate.
Model	ID	Family	𝑬𝟏	𝑬𝟐,𝐦𝐞𝐚𝐧	𝑬𝟐,𝐩𝟗𝟎	𝑬𝟏𝟒	𝑬𝟏𝟓
Qwen3-8B	M1	Flat	9.7M	2,075	4,602	2,191	7.30
M2	Flat	2.7M	1,747	3,580	525	1.75
M3	Text	4.5M	4,500	9,000	16,674	55.58
M4	Struct	1.6M	1,828	4,233	67,702	225.67
M5	Struct	32.0M	8,500	18,000	56,162	187.21
M6	Hier	27.5M	19,956	5,195	5,987	19.96
M8	Refine	22.7M	1,869	4,392	30,855	102.85
M9	Weight	80.0M	2,400	5,000	15,451	51.50
M10	Act	60K	3,744	5,539	1,983	6.61
M11	Act	80K	1,900	4,200	4,382	14.61
Qwen3-32B-AWQ	M1	Flat	9.7M	7,413	18,100	3,600	12.00
M2	Flat	2.7M	7,307	18,824	2,193	7.31
M3	Text	4.5M	16,076	32,153	20,226	67.42
M4	Struct	1.6M	7,070	18,019	70,200	234.00
M5	Struct	32.0M	18,000	35,000	58,225	194.08
M6	Hier	27.5M	20,271	17,816	6,082	20.27
M8	Refine	23.4M	6,149	15,192	32,034	106.78
M9	Weight	320M	8,500	19,000	51,511	171.70
M10	Act	60K	14,650	28,022	6,894	22.98
M11	Act	80K	7,000	17,000	15,717	52.39
Gemma-4-26B	M1	Flat	9.7M	4,149	8,550	2,546	8.49
M2	Flat	2.7M	3,372	6,501	1,012	3.37
M3	Text	4.5M	8,998	17,996	9,534	31.78
M4	Struct	1.6M	3,805	9,027	69,116	230.39
M5	Struct	32.0M	12,000	23,000	57,280	190.93
M6	Hier	27.5M	17,189	9,007	5,157	17.19
M8	Refine	21.7M	3,124	6,540	31,140	103.80
M10	Act	60K	3,340	3,945	2,073	6.91
M11	Act	80K	–	–	5,550	18.50

Table 12: LongMemEval-S token usage (E5–E13).
Model	ID	Family	𝑬𝟓	𝑬𝟔	𝑬𝟕	𝑬𝟏𝟐	𝑬𝟏𝟑	𝑷𝟓
Qwen3-8B	M1	Flat	0	0	101K	0	0	1.00
M2	Flat	0	0	153K	0	0	1.00
M3	Text	437K	671	5.0M	5.3M	731	4.50
M4	Struct	4.4M	1310	108K	4.4M	1310	6.20
M5	Struct	10.3M	2371	375K	10.4M	2431	3.80
M6	Hier	280K	80	46K	280K	80	2.80
M8	Refine	1.8M	1752	21K	2.78M	3504	12.50
M9	Weight	–	–	–	–	–	–
M10	Act	–	–	1.9M	–	–	–
M11	Act	–	–	250K	–	–	–
Qwen3-32B-AWQ	M1	Flat	0	0	101K	0	0	1.00
M2	Flat	0	0	153K	0	0	1.00
M3	Text	437K	671	5.0M	5.3M	731	4.50
M4	Struct	4.4M	1310	108K	4.4M	1310	6.20
M5	Struct	10.4M	2369	385K	10.4M	2429	3.80
M6	Hier	280K	80	46K	280K	80	2.80
M8	Refine	1.8M	1752	20K	2.78M	3504	12.50
M9	Weight	–	–	–	–	–	–
M10	Act	–	–	1.9M	–	–	–
M11	Act	–	–	250K	–	–	–
Gemma-4-26B	M1	Flat	0	0	104K	0	0	1.00
M2	Flat	0	0	158K	0	0	1.00
M3	Text	437K	671	5.0M	5.3M	731	4.50
M4	Struct	4.4M	1310	111K	4.4M	1310	6.20
M5	Struct	10.4M	2377	379K	10.4M	2437	3.80
M6	Hier	280K	80	46K	280K	80	2.80
M8	Refine	1.9M	1752	23K	2.98M	3504	12.50
M9	Weight	–	–	–	–	–	–
M10	Act	–	–	422K	–	–	–
M11	Act	–	–	250K	–	–	–

Table 13: Table for different substrate and model ALFWorld-unseen Performance.
Model	Sub.	Family	P8 TSR	Steps	P10 Steps∣𝒮	Pavg
Qwen3-8B	–	NoMem	5.7	47.0	12.2	13.5
M1	Flat	5.2	47.7	5.9	6.6
M2	Flat	6.7	47.0	6.0	9.4
M3	Text	5.2	47.8	8.3	8.4
M4	Struct	7.5	46.9	7.9	8.5
M5	Struct	9.0	47.1	17.4	12.5
M6	Hier	4.5	48.4	15.0	8.3
M7	Refine	7.5	47.3	13.7	9.6
M8	Refine	8.2	47.4	18.5	9.5
M9	Weight	3.0	49.1	19.5	3.2
M11	Act	11.9	45.7	14.2	14.5
Qwen3-32B-AWQ	–	NoMem	22.4	41.6	11.3	23.7
M1	Flat	27.6	39.6	9.6	16.2
M2	Flat	21.6	41.4	10.8	15.4
M3	Text	26.9	39.4	9.9	21.8
M4	Struct	26.9	39.5	12.4	24.3
M5	Struct	23.1	41.1	11.4	23.0
M6	Hier	23.9	40.2	9.9	22.8
M7	Refine	32.1	37.5	11.1	29.9
M8	Refine	22.4	41.6	12.4	21.3
M9	Weight	29.9	38.7	12.1	27.4
M11	Act	26.9	39.5	12.3	24.7
Gemma-4-26B	–	NoMem	7.5	47.1	11.3	17.3
M1	Flat	10.4	46.3	15.0	16.3
M2	Flat	9.7	46.3	11.5	17.2
M3	Text	10.4	46.3	14.8	14.9
M4	Struct	8.2	47.1	14.5	13.9
M5	Struct	4.5	48.1	8.5	9.4
M6	Hier	10.4	46.0	11.4	18.1
M7	Refine	8.2	46.7	10.1	17.8

Table 14: Table for different substrate and model ALFWorld-unseen Latency & Storage.
Model	Sub.	Family	𝑬𝟏	𝑬𝟐	𝑬𝟐,p90	𝑬𝟑	𝑬𝟒	𝑬𝟏𝟒	𝑬𝟏𝟓
Qwen3-8B	–	NoMem	0	12,728	13,200	–	–	65,392	488
M1	Flat	2.17M	14,086	16,104	188	220	90,048	672
M2	Flat	1.49M	13,747	15,692	0	1	86,698	647
M3	Text	1.43M	13,655	16,318	1,620	754	87,502	653
M4	Struct	942K	12,903	14,843	6,134	9	81,070	605
M5	Struct	532K	13,167	15,702	5,021	9,412	83,080	620
M6	Hier	25.1M	13,539	15,641	624	191	87,904	656
M7	Refine	3.90M	13,297	15,988	2,519	175	84,286	629
M8	Refine	5.27M	13,069	15,300	2,521	173	83,080	620
M9	Weight	15.3M	18,757	22,002	1,400	–	123,414	921
M11	Act	6.9K	12,728	13,500	–	–	74,102	553
Qwen3-32B-AWQ	–	NoMem	0	20,100	20,300	–	–	78,122	583
M1	Flat	2.17M	20,500	31,100	192	644	144,720	1,080
M2	Flat	1.49M	20,300	30,800	0	1	146,060	1,090
M3	Text	1.43M	20,900	30,300	1,603	1,087	127,300	950
M4	Struct	942K	20,600	28,600	6,284	12	109,344	816
M5	Struct	532K	20,736	27,400	5,154	9,477	114,034	851
M6	Hier	25.1M	20,400	26,800	642	600	109,478	817
M7	Refine	3.90M	20,200	27,200	2,575	165	95,944	716
M8	Refine	5.27M	20,500	28,000	2,571	164	118,724	886
M9	Weight	41.9M	15,825	21,400	3,800	–	82,008	612
M11	Act	6.9K	20,100	22,000	–	–	104,922	783
Gemma-4-26B	–	NoMem	0	16,060	16,400	–	–	89,914	671
M1	Flat	2.17M	16,800	21,600	195	184	105,324	786
M2	Flat	1.49M	16,500	21,400	0	1	100,902	753
M3	Text	1.43M	16,857	20,466	1,649	1,009	104,654	781
M4	Struct	942K	20,007	22,517	6,089	11	126,228	942
M5	Struct	532K	15,564	19,906	5,089	9,358	100,366	749
M6	Hier	25.1M	16,114	21,099	619	173	99,294	741
M7	Refine	3.90M	16,866	21,107	2,542	161	105,592	788

Table 15: Table for different substrate ALFWorld-unseen Token Accounting.
Sub.	Family	𝑬𝟓	𝑬𝟔	𝑬𝟕	𝑬𝟖	𝑬𝟗	𝑬𝟏𝟎	𝑬𝟏𝟏	𝑬𝟏𝟐	𝑬𝟏𝟑
M1	Flat	0	0	391K	0	0	0	0	0	0
M2	Flat	0	0	3.66M	0	0	0	0	0	0
M3	Text	372K	200	8.14M	1.84M	1,340	0	0	2.21M	1,540
M4	Struct	1.62M	199	5.05M	0	0	480K	380	2.10M	579
M5	Struct	1.42M	385	261	1.21M	770	0	0	2.63M	1,155
M6	Hier	357K	460	235K	0	0	84.6K	92	442K	552
M7	Refine	398K	200	183K	0	0	0	0	398K	200
M8	Refine	931K	577	13.5K	0	0	277K	380	1.21M	957
M9	Weight	–	–	8.7K	–	–	–	–	–	–
M11	Act	–	–	118K	–	–	–	–	–	–

Table 16: Table for different substrate and model BigCodeBench-Hard Performance & Quality.
Model	Sub.	Family	Pass@1 ↑	𝑷𝐚𝐯𝐠 ↑	𝑷𝟓
Qwen3-8B	–	NoMem	8.1	12.4	–
M1	Flat	9.5	20.1	1.000
M2	Flat	10.1	11.9	1.000
M3	Text	8.1	12.4	1.964
M4	Struct	10.1	20.4	0.982
M5	Struct	15.5	17.1	0.817
M6	Hier	12.2	22.2	0.820
M7	Refine	12.2	16.9	5.705
M8	Refine	14.9	24.0	4.432
M9	Weight	7.4	4.6	–
M11	Act	13.5	23.6	–
Qwen3-32B-AWQ	–	NoMem	17.6	25.6	–
M1	Flat	18.2	24.2	1.000
M2	Flat	19.6	21.8	1.000
M3	Text	18.2	24.5	1.995
M4	Struct	17.6	22.4	0.910
M5	Struct	16.2	16.6	0.786
M6	Hier	16.2	16.8	0.799
M7	Refine	17.6	13.5	2.261
M8	Refine	13.5	23.0	3.575
M9	Weight	14.2	15.5	–
M11	Act	13.5	13.8	–
Gemma-4-26B	–	NoMem	14.2	19.5	–
M1	Flat	18.2	26.4	1.000
M2	Flat	19.6	27.0	1.000
M3	Text	14.2	19.5	2.444
M4	Struct	18.9	26.9	0.982
M5	Struct	20.9	28.4	0.813
M6	Hier	18.9	26.0	0.813
M7	Refine	16.9	25.1	1.102

Table 17: Table for different substrate and model BigCodeBench-Hard Latency & Storage.
Model	Sub.	Family	𝑬𝟏	𝑬𝟐	𝑬𝟐,p90	𝑬𝟑	𝑬𝟒	𝑬𝟏𝟒	𝑬𝟏𝟓
Qwen3-8B	–	NoMem	150K	3,800	4,940	–	–	592	4
M1	Flat	1.32M	4,976	6,631	199	183	776	5.2
M2	Flat	245K	4,853	6,600	0.1	3	718	4.9
M3	Text	305K	5,300	6,890	6,853	46	784	5.3
M4	Struct	150K	4,642	6,432	137	138	747	5.0
M5	Struct	239K	28,273	36,700	8,779	12,432	4,184	28.3
M6	Hier	3.42M	4,764	6,234	1,444	184	994	6.7
M7	Refine	990K	4,638	6,256	9,029	183	2,519	17.0
M8	Refine	1.22M	5,053	6,825	10,440	197	2,836	19.2
M9	Weight	14.6M	12,700	16,510	–	–	1,924	13
M11	Act	151K	16,800	21,840	–	–	2,516	17
Qwen3-32B-AWQ	–	NoMem	150K	3,900	5,070	–	–	592	4
M1	Flat	1.32M	5,000	6,500	256	225	740	5
M2	Flat	245K	5,000	6,500	0.1	3	740	5
M3	Text	296K	5,400	7,020	9,309	1,843	1,184	8
M4	Struct	150K	4,900	6,370	12,460	376	2,960	20
M5	Struct	249K	31,221	28,239	7,888	14,723	4,621	31.2
M6	Hier	3.50M	4,400	5,720	1,995	220	1,036	7
M7	Refine	1.93M	4,300	5,590	12,994	195	3,108	21
M8	Refine	1.14M	4,600	5,980	12,447	186	3,256	22
M9	Weight	38.0M	14,600	18,980	–	–	2,220	15
M11	Act	161K	21,300	27,690	–	–	3,152	21.3
Gemma-4-26B	–	NoMem	150K	19,800	25,740	–	–	2,960	20
M1	Flat	1.32M	21,916	28,533	213	289	3,286	22.2
M2	Flat	245K	20,602	27,860	0.0	2	3,049	20.6
M3	Text	272K	23,400	30,420	18,378	30	3,463	23.4
M4	Struct	150K	22,375	31,073	173	92	3,372	22.8
M5	Struct	240K	45,595	47,436	8,194	13,087	6,748	45.6
M6	Hier	3.47M	24,310	33,614	2,717	251	4,141	28.0
M7	Refine	3.13M	24,149	31,513	8,921	317	5,460	36.9

Table 18: Table for different substrate and model BigCodeBench-Hard Token Accounting.
Model	Sub.	Family	𝑬𝟓	𝑬𝟔	𝑬𝟕	𝑬𝟖	𝑬𝟗	𝑬𝟏𝟎	𝑬𝟏𝟏	𝑬𝟏𝟐	𝑬𝟏𝟑
Qwen3-8B	–	NoMem	–	–	4.75M	–	–	–	–	–	–
M1	Flat	0	0	2.29M	0	0	0	0	0	0
M2	Flat	0	0	2.26M	0	0	0	0	0	0
M3	Text	55.9K	36	2.80M	2.84M	148	0	0	2.90M	184
M4	Struct	0	0	2.16M	0	0	0	0	0	0
M5	Struct	0	0	40.7K	116K	148	0	0	116K	148
M6	Hier	44.7K	82	1.14M	0	0	11.5K	16	56.2K	98
M7	Refine	244K	200	593K	0	0	0	0	244K	200
M8	Refine	308K	200	550K	0	0	146K	200	454K	400
M9	Weight	–	–	0	–	–	–	–	–	–
M11	Act	–	–	1.44M	–	–	–	–	–	–
Qwen3-32B-AWQ	–	NoMem	–	–	4.75M	–	–	–	–	–	–
M1	Flat	0	0	2.29M	0	0	0	0	0	0
M2	Flat	0	0	2.26M	0	0	0	0	0	0
M3	Text	55.6K	36	2.76M	2.81M	148	0	0	2.86M	184
M4	Struct	524K	199	286K	0	0	0	0	524K	199
M5	Struct	0	0	41.7K	121K	148	0	0	121K	148
M6	Hier	45.1K	86	575K	0	0	13.5K	24	58.6K	110
M7	Refine	245K	200	91.6K	0	0	0	0	245K	200
M8	Refine	293K	200	84.8K	0	0	148K	200	441K	400
M9	Weight	–	–	0	–	–	–	–	–	–
M11	Act	–	–	1.48M	–	–	–	–	–	–
Gemma-4-26B	–	NoMem	–	–	4.75M	–	–	–	–	–	–
M1	Flat	0	0	2.29M	0	0	0	0	0	0
M2	Flat	0	0	2.26M	0	0	0	0	0	0
M3	Text	52.2K	36	2.26M	2.30M	148	0	0	2.35M	184
M4	Struct	0	0	2.16M	0	0	0	0	0	0
M5	Struct	0	0	40.1K	177K	148	0	0	177K	148
M6	Hier	44.4K	82	1.14M	0	0	12.6K	24	57.0K	106
M7	Refine	161K	200	901K	0	0	0	0	161K	200

为什么重要

这为构建长期运行的编程助手、个人AI伴侣或机器人智能体提供了选择记忆架构的实证依据。它表明通用的智能体记忆系统必须根据任务类型和对话历史长度动态切换存储策略,而不能固定使用某一种方式。

本文术语

记忆载体(memory substrate) · 智能体存放记忆所用的底层介质,如向量数据库、知识图谱或摘要树
稠密/稀疏向量索引 · 把文本转成数值向量做相似度检索(稠密),或基于关键词匹配检索(稀疏)
知识图谱(结构化存储) · 把实体和关系以节点边形式连接存储,便于多跳推理
top-k检索 · 只取与问题最相关的k条存储记忆来生成答案
LLM裁判评分 · 用另一个大模型(如GPT-4o-mini)自动给答案质量打分,替代人工评审

无法转载的图表

Figure 1: Evaluation landscape of 52 memory-augmented LLM systems (2023–2026). (a) Benchmark adoption: 62% of usage concentrates on LoCoMo and LongMemEval (b) Metric coverage: every system reports accuracy but only 21% report any efficiency metric; 81% of systems use GPT-family backbones. (c) Breadth and efficiency: half of systems use a single benchmark with no efficiency metrics; no system simultaneously achieves broad coverage and comprehensive efficiency reporting.
Figure 2: Overview of the harness evaluation. 11 memory methods (M1–M11) spanning seven substrate families are evaluated on identical interaction histories across four benchmarks, instrumented with 26 metrics in two families (performance and efficiency).
Figure 3: Performance vs. latency across four benchmarks on Qwen3-32B-AWQ. (a) LoCoMo and (b) LongMemEval-S: absolute P4. (c) ALFWorld-unseen and (d) BigCodeBench-Hard: Δ-performance vs. latency overhead anchored at NoMem (No memory baseline).
Figure 4: Retrieval breadth k scales oppositely on long-context QA versus sequential decision-making. Top: LoCoMo (Qwen3-8B, n=1986, k∈{1,2,5,10,20}). Bottom: ALFWorld-unseen (Qwen3-32B-AWQ, n=134, k∈{1,2,3,4,5}). green when the trend is in the preferred direction, red otherwise. No Mem stands for no memory baseline.
Figure 5: Response-token attention under M1 retrieval for (a) LoCoMo (b) ALFWorld.
Figure 6: MAB CR scalability on Qwen3-32B-AWQ: P4 (bars, left) and per-query latency (lines, right, log) at 6K, 32K, 262K tokens.

在原文中查看图表 →

论文原文摘要（英文）

Memory is becoming core infrastructure for long-horizon LLM agents, yet existing evaluations offer limited guidance on which memory substrate, namely the underlying medium in which memory is represented and stored, should be used under different operating regimes. We present a controlled harness evaluation of memory substrates for memory-augmented agents, covering dense and sparse indices, text records, structural stores, hierarchical stores, refinement-based memories, parametric updates, and activation-compatible context mechanisms. Across three backbone models and four benchmark suites spanning user-centric question answering and agent-centric decision-making, we instrument 26 performance and efficiency metrics under a unified harness. Our results show that no single substrate consistently dominates: broad retrieval benefits long-context factual QA, while excessive retrieval can harm sequential decision-making by shifting attention away from action-critical context. Scalability introduces a further routing axis, as substrates that perform well at moderate history lengths can become costly or brittle at longer horizons. These findings motivate substrate routing as a necessary component of adaptive agent memory systems and provide empirical guidance for designing efficient, reliable, and regime-aware long-term memory for LLM agents. Code will be made available upon acceptance.

作者 · Wei-Chieh Huang

在 arXiv 阅读

Harness the Memory: A Holistic Evaluation of Memory Substrates in Memory Agents

把AI智能体的11种记忆存储方式放在同一条件下比拼,发现没有一种能通吃所有场景

他们做了什么

为什么重要

本文术语

无法转载的图表

最新论文

METAL MEDIA 最新报道