K-文化的一切——从回归到 K-美妆，发送到您的邮箱订阅邮件

METAL MEDIA

Thought-Level Beam Search for Reasoning

arXiv:2608.080202026-08-10

Gambit在推理模型生成多条答案的过程中,实时把算力集中投给最有希望的那条

大型推理模型常靠并行生成多条推理路径再多数投票(Self-Consistency)来提高准确率,但大部分路径最终都是错的,浪费了大量算力。Gambit在生成过程中定期用轻量打分器读取隐藏状态给各条推理路径打分,砍掉分数最低的几条,同时立刻从分数最高的前缀分支出新的路径,让GPU利用率始终维持在高位。在相同硬件预算下,相比纯剪枝方法,它在HMMT-24上准确率最多提高6.7个百分点、在AIME-25上提高3.3个百分点,完成路径的吞吐量提高一倍以上,相比标准并行采样总token消耗最多减少68.5%。

METAL MEDIA 解读图

Gambit如何主动重新分配算力

证据状态已报告实测结果

  1. 固定容量启动针对一个问题,C条推理路径同时开始生成,填满固定的硬件容量
  2. 周期性打分每隔Δ步,轻量打分器读取每条路径的隐藏状态,对所有活跃路径排序
  3. 零和式剪枝与分支终止分数最低的K条路径,同时通过KV-cache继承从分数最高的K条前缀分支出新路径,活跃路径数始终保持为C
  4. 完成与加权投票完成的路径按各自分数加权,参与多数投票决定最终答案
  5. 实测效果相比STEP准确率最多提升6.7个百分点,相比并行采样token消耗最多减少68.5%,吞吐量提升一倍以上
这是 METAL MEDIA 制作的解读图，并非论文作者提供的原图。

他们做了什么

  1. 把大型推理模型的推理时算力扩展问题重新定义为一个分配问题:不是要花多少算力,而是要把算力投向哪条部分推理路径。
  2. 指出现有两种范式的两难困境:并行采样(Self-Consistency)把每条路径独立对待,导致显存(KV-cache)迅速耗尽;而只做剪枝的方法虽然释放了显存,却让GPU闲置,没有真正改变输出分布。
  3. Gambit每隔Δ步用一个读取隐藏状态的轻量打分模型给所有在跑的推理路径打分,砍掉分数最低的K条,同时立刻从分数最高的K条前缀分支出新的续写,让活跃路径数量始终保持恒定(零和式重新分配)。
  4. 新分支的路径通过前缀缓存直接继承父路径的KV-cache,不用从头重新计算,因此能大幅节省token和显存开销。
  5. 在Qwen3-4B、DeepSeek-R1-8B、Phi-4-14B等多个模型以及AIME、HMMT、GPQA-Diamond等基准上的实测显示,Gambit在准确率、吞吐量、token效率三方面同时全面优于对比方法。
Figure 1: Reasoning topologies for test-time compute allocation. (Left) Parallel sampling treats traces as independent trials, leading to wrong answers in majority. (Center) Score-based pruning terminates unpromising paths but leaves the freed capacity idle, failing to actively shift the sampling distribution. (Right) Gambit actively reallocates compute: when low-scoring traces are pruned, it immediately branches from high-quality prefixes to amplify correct reasoning and saturate the compute budget.
Figure 1: Reasoning topologies for test-time compute allocation. (Left) Parallel sampling treats traces as independent trials, leading to wrong answers in majority. (Center) Score-based pruning terminates unpromising paths but leaves the freed capacity idle, failing to actively shift the sampling distribution. (Right) Gambit actively reallocates compute: when low-scoring traces are pruned, it immediately branches from high-quality prefixes to amplify correct reasoning and saturate the compute budget.
Table 1: End-to-end task accuracy and token consumption per question (×106) at N=256. Δ: relative token reduction (%) vs. SC@256. Bold: best accuracy / lowest tokens per dataset.
MethodAIME-25AIME-26HMMT-24HMMT-25GPQA
Tok. (Δ)AccTok. (Δ)AccTok. (Δ)AccTok. (Δ)AccTok. (Δ)Acc
Qwen3-4B-Thinking
SC@2566.0286.75.8286.77.6250.86.8665.02.2868.2
Slim-SC3.93 (-34.7)86.73.83 (-34.2)85.04.01 (-47.4)51.73.72 (-45.8)65.81.66 (-27.2)65.6
DeepConf3.27 (-45.7)90.03.19 (-45.2)86.74.28 (-43.8)58.34.15 (-39.5)66.71.52 (-33.3)67.6
STEP3.96 (-34.2)86.73.51 (-39.7)87.54.02 (-47.2)61.73.88 (-43.4)66.72.09 (-8.3)66.9
Gambit (Ours)3.07 (-49.0)90.03.45 (-40.7)88.33.00 (-60.6)65.03.49 (-49.1)67.51.83 (-19.7)70.2
DeepSeek-R1-8B
SC@2566.7683.36.8583.38.4755.87.6570.02.9267.1
Slim-SC5.87 (-13.2)83.36.08 (-11.2)83.37.32 (-13.6)55.86.93 (-9.4)70.72.26 (-22.6)67.1
DeepConf3.75 (-44.5)81.73.67 (-46.4)84.24.34 (-48.8)60.03.97 (-48.1)71.71.68 (-42.5)68.7
STEP3.71 (-45.1)83.33.66 (-46.6)83.34.15 (-51.0)63.34.08 (-46.7)75.82.44 (-16.4)68.2
Gambit (Ours)4.21 (-37.7)85.84.09 (-40.3)85.03.05 (-64.0)65.63.81 (-50.2)75.82.21 (-25.3)68.2
Microsoft-Phi
SC@2564.2486.74.2088.55.4356.75.5673.33.0576.3
Slim-SC3.43 (-19.1)85.03.50 (-16.7)86.74.72 (-13.1)55.74.48 (-19.4)74.22.24 (-26.6)72.3
DeepConf2.56 (-39.6)85.82.75 (-34.5)86.72.86 (-47.3)57.53.02 (-45.7)74.21.61 (-47.2)74.8
STEP2.39 (-43.6)87.52.44 (-41.9)90.02.75 (-49.4)56.72.78 (-50.0)75.02.10 (-31.1)76.7
Gambit (Ours)1.76 (-58.5)88.31.86 (-55.7)90.01.72 (-68.3)58.31.75 (-68.5)75.81.59 (-47.9)77.1
Figure 2: Branching from a high-quality prefix versus independent sampling on a hard AIME 2025 problem (Q27). Branching 64 continuations from the top-ranked prefix achieves 87.5% pass@1 (14× over 6.2% baseline) while sharing the parent’s KV-cache (green region) halves memory consumption.
Figure 2: Branching from a high-quality prefix versus independent sampling on a hard AIME 2025 problem (Q27). Branching 64 continuations from the top-ranked prefix achieves 87.5% pass@1 (14× over 6.2% baseline) while sharing the parent’s KV-cache (green region) halves memory consumption.
Table 2: Impact of scorer architecture on end-to-end downstream accuracy. Gambit consistently outperforms the pruning-only baseline (STEP) across different reward models and base LLMs. By actively reallocating compute to promising prefixes, Gambit effectively unlocks the potential of the guidance signal, whereas purely subtractive methods remain fundamentally bottlenecked by their inability to generate new traces.
Base ModelSystemAIME-25AIME-26HMMT-24HMMT-25GPQA
DeepSeek-R1-0528 -Qwen3-8BSTEP + SeqScorer83.384.261.775.866.7
Gambit + SeqScorer88.5 (+5.2)86.7 (+2.5)69.4 (+7.7)76.7 (+0.9)67.7 (+1.0)
Qwen3-4B -Thinking-2507STEP + SeqScorer86.786.760.065.067.1
Gambit + SeqScorer90.0 (+3.3)87.5 (+0.8)61.7 (+1.7)65.8 (+0.8)68.2 (+1.1)
Figure 3: GPU memory utilization and latency profiles during reasoning with batch_size=256 traces on HMMT-25 Q7. Parallel sampling exhausts KV-cache capacity, inflating latency by ∼3×. Pruning-only STEP (Liang et al. 2026) eliminates queueing but steadily loses concurrency. Gambit maintains high utilization throughout at ∼1.1× latency.
Figure 3: GPU memory utilization and latency profiles during reasoning with batch_size=256 traces on HMMT-25 Q7. Parallel sampling exhausts KV-cache capacity, inflating latency by ∼3×. Pruning-only STEP (Liang et al. 2026) eliminates queueing but steadily loses concurrency. Gambit maintains high utilization throughout at ∼1.1× latency.
Table 3: Latency decomposition for Gambit with Qwen3-4B on AIME-26 with a batch size of N=256. Algorithmic overhead accounts for less than 1% of total execution time, confirming that the beam search mechanism operates efficiently without introducing system bottlenecks.
Execution ComponentTime (s)% Wall Clock
GPU Compute (Forward Pass & Sampling)37,851.9799.03%
Beam-search Overhead (Scoring & Tree Mgmt)370.760.97%
Total Wall Clock38,222.73100.00%
Figure 4: End-to-end pipeline of Gambit on an AIME problem (capacity C=5, swap size K=2). During warmup, C=5 parallel traces are evaluated (color intensity denotes running score s¯∈[0,1]). Every Δ steps, a tournament ranks the active traces: the K=2 lowest-scoring traces are pruned (×) and replaced by new branches spawned from the highest-scoring prefixes via prefix caching (green arrows). This zero-sum reallocation maintains exactly C active traces throughout generation. Upon completion, answers are aggregated via a score-weighted majority vote, correctly selecting answer 29 (∑s¯=1.78 vs. 1.43).
Figure 4: End-to-end pipeline of Gambit on an AIME problem (capacity C=5, swap size K=2). During warmup, C=5 parallel traces are evaluated (color intensity denotes running score s¯∈[0,1]). Every Δ steps, a tournament ranks the active traces: the K=2 lowest-scoring traces are pruned (×) and replaced by new branches spawned from the highest-scoring prefixes via prefix caching (green arrows). This zero-sum reallocation maintains exactly C active traces throughout generation. Upon completion, answers are aggregated via a score-weighted majority vote, correctly selecting answer 29 (∑s¯=1.78 vs. 1.43).
Table 4: Systems-level efficiency at n=256 on a single B300-275 GB. Token consumption is reported in thousands (K); latency in wall-clock seconds per question. Bold indicates the best result per column within each model–metric block.
AIME-25AIME-26HMMT-24HMMT-25GPQA
Qwen3-4B-Thinking-2507
Tokens (K)SC6,0235,8177,6226,8572,276
Slim-SC3,9303,8294,0103,7231,659
DeepConf3,2663,1924,2784,1481,516
STEP3,9603,5064,0233,8842,089
Gambit (Ours)3,0713,4493,0023,4931,826
Latency (s)SC3,4653,3385,3584,313593
Slim-SC1,9051,8412,4121,995555
DeepConf2,0952,1883,2312,948605
STEP1,3901,2322,0161,507786
Gambit (Ours)1,3501,1771,9122,179527
DeepSeek-R1-0528-Qwen3-8B
Tokens (K)SC6,7646,8498,4657,6522,919
Slim-SC5,8746,0807,3236,9332,256
DeepConf3,7533,6664,3373,9721,679
STEP3,7113,6614,1474,0772,444
Gambit (Ours)4,2114,0893,0533,8062,211
Latency (s)SC4,0784,1075,6404,873880
Slim-SC3,4623,5645,0714,342771
DeepConf2,6922,5823,2562,800744
STEP1,5191,4691,9021,715730
Gambit (Ours)1,5711,8422,0142,169702
Phi-4
Tokens (K)SC4,2434,1985,4265,5593,050
Slim-SC3,4263,5034,7164,4822,242
DeepConf2,5572,7492,8603,0221,608
STEP2,3902,4352,7542,7802,097
Gambit (Ours)1,7561,8551,7171,7521,590
Latency (s)SC2,7602,6363,8484,0461,730
Slim-SC2,1232,1663,0692,7041,048
Figure 5: Efficiency vs. Accuracy trade-offs over all benchmarks. (a) Average Latency vs. Accuracy. (b) Average Token Consumption vs. Accuracy. Gambit (Ours) dominates the efficiency frontier, achieving the highest accuracy with the lowest overall token footprint and highly competitive wall-clock latency.
Figure 5: Efficiency vs. Accuracy trade-offs over all benchmarks. (a) Average Latency vs. Accuracy. (b) Average Token Consumption vs. Accuracy. Gambit (Ours) dominates the efficiency frontier, achieving the highest accuracy with the lowest overall token footprint and highly competitive wall-clock latency.

研究结果

  • 在相同硬件限制下,Gambit相比纯剪枝基线STEP,在HMMT-24上准确率最多提高6.7个百分点,在AIME-25上提高3.3个百分点。
  • 相比标准并行采样(Self-Consistency),Gambit将总token消耗最多降低68.5%(Phi-4在HMMT-25上:175万 vs 556万token),完成路径的吞吐量提高一倍以上(Qwen3-4B上为0.216对0.098)。
  • 在4B模型上,Gambit在AIME-25上与经过精细校准的DeepConf打平(均为90.0%),同时在HMMT-24(+6.7个百分点)和GPQA(+2.6个百分点)上更优。
  • 束搜索机制本身的算法开销经测量仅占总运行时间的不到1%(0.97%),证实其不会造成系统瓶颈。
  • 在一道AIME 2025题目的案例分析中,即便把并行采样的样本数翻倍到512条,标准方法仍未找到正确答案,而Gambit用少得多的生成token就得到了正确答案。
Figure 6: Trace throughput on AIME-26. Gambit effectively doubles the completion rate.
Figure 6: Trace throughput on AIME-26. Gambit effectively doubles the completion rate.

可应用场景

  • 可作为竞赛级数学或研究生级科学问题等正确路径稀少、推理链很长的任务中的推理时算力分配策略。
  • 对使用vLLM等推理服务框架、希望同时缓解KV-cache显存压力和GPU闲置问题的推理服务运营者具有参考价值。
  • 已经在用STEP、DeepConf等轻量内部打分信号的系统,可以考虑用主动分支重新分配算力取代单纯剪枝的思路。
Figure 7: Distribution of unique tokens generated per completed trace on AIME-2026. By branching from shared high-quality KV-cache prefixes, Gambit drastically reduces the median number of new tokens required to explore alternative reasoning pathways compared to SC and STEP.
Figure 7: Distribution of unique tokens generated per completed trace on AIME-2026. By branching from shared high-quality KV-cache prefixes, Gambit drastically reduces the median number of new tokens required to explore alternative reasoning pathways compared to SC and STEP.

局限与待验证事项

  • 实测范围仅限于AIME、HMMT、GPQA-Diamond等数学科学竞赛类基准,以及Qwen3-4B、DeepSeek-R1-8B、Phi-4-14B等特定开源权重模型,尚未报告在其他领域或更大规模模型上的验证。
  • 效果依赖打分函数的质量,论文只比较了STEP的现成MLP打分器和作者自行训练的序列打分器两种。
  • 由于早期推理信号被认为不稳定,系统在预热阈值(w=12K token)之前不进行打分和检查点保存,这一阈值设置是否适用于其他类型问题未做单独验证。
  • 结果均在单块NVIDIA B300 GPU上测得,在多GPU或其他硬件世代上的复现情况论文中未报告。
Figure 9: Pairwise ranking accuracy vs. fraction of reasoning steps used (k%). Higher accuracy indicates better separation of correct and incorrect traces given only a partial prefix.
Figure 9: Pairwise ranking accuracy vs. fraction of reasoning steps used (k%). Higher accuracy indicates better separation of correct and incorrect traces given only a partial prefix.

为什么重要

部署推理型AI模型最大的成本就是为了提高准确率而反复生成候选答案所消耗的GPU时间和显存。Gambit展示了在相同硬件条件下如何同时获得更高准确率、更快完成速度和更低token消耗,这对任何需要优化推理模型运行成本的人都有直接参考价值。

Figure 10: Hyperparameter sensitivity of Gambit on AIME-25 with DeepSeek-R1-0528-8B (batch size 256, C=256). Each panel varies one parameter while fixing the others to the selected configuration (K=16,Δ=200,w=12​K,r=0.9). The dashed line denotes the SC@256 baseline (83.3%). The selected configuration (red circle) lies within a broad high-performing regime, consistently outperforming the baseline.
Figure 10: Hyperparameter sensitivity of Gambit on AIME-25 with DeepSeek-R1-0528-8B (batch size 256, C=256). Each panel varies one parameter while fixing the others to the selected configuration (K=16,Δ=200,w=12​K,r=0.9). The dashed line denotes the SC@256 baseline (83.3%). The selected configuration (red circle) lies within a broad high-performing regime, consistently outperforming the baseline.

本文术语

  • Self-Consistency · 对同一问题独立生成多条推理路径,再用多数投票选出最终答案的方法
  • KV-cache · Transformer模型存储此前计算过的键值向量的显存空间,供后续生成token时复用
  • 束搜索(beam search) · 在搜索过程中只保留分数最高的一部分候选路径、舍弃其余的策略;本文将其应用在'思考步骤'层面而非单个token层面
  • 剪枝(pruning) · 在推理过程中提前终止被判定希望渺茫的路径,以节省算力
  • 隐藏状态(hidden state) · 模型内部层产生的中间表示向量,这里被用作轻量评估某条推理路径质量的信号

论文原文摘要（英文）

Test-time compute scaling is a primary driver of performance in large reasoning models (LRMs), but extreme inefficiency bounds current approaches, shifting the critical question from how much compute to spend, to where to allocate it. We formalize test-time reasoning as a constrained compute allocation problem over partial trajectories. Under a fixed hardware budget, existing paradigms fail to actively allocate the compute to the most promising partial progress: traditional parallel sampling treats traces independently and induces severe memory bottlenecks, while subtractive pruning starves hardware and fails to actively and sufficiently shift the output distribution. To overcome this dichotomy, we introduce Gambit, an inference algorithm that executes thought-level beam search. By periodically pruning unpromising trajectories and immediately branching from high-quality prefixes, Gambit dynamically concentrates compute onto the most promising reasoning traces via a light-weight scorer probing hidden states while maintaining continuous high hardware utilization. Extensive evaluations across multiple models and benchmarks demonstrate that Gambit strictly dominates existing baselines. Under identical hardware constraints, our method yields up to a +6.7\% absolute accuracy gain on HMMT-24 and +3.3\% on AIME-25 over pruning baselines, delivers >2times higher throughput on trace completion, and reduces total token consumption by up to 68.5\% relative to standard parallel sampling.

作者 · Lijie Yang

在 arXiv 阅读

最新论文

全部论文 →

METAL MEDIA 最新报道

图片来源： Lijie Yang et al., arXiv:2608.08020, CC BY-SA 4.0