K-文化的一切——从回归到 K-美妆，发送到您的邮箱订阅邮件

METAL MEDIA

The Problem Is the Problem: Towards Scalable Mathematical Discovery

arXiv:2608.169772026-08-16

让AI从海量论文里自动寻找未解决的数学问题、尝试求解并层层筛选,把人力集中在最后少数值得评审的成果上

这篇论文指出,当前AI辅助数学研究的流程里,人的精力集中在两头——挑选要研究的问题,以及审阅AI给出的结果——这两处正成为瓶颈。作者提出让数学家只需指定一个感兴趣的研究方向,系统FAR(Find, Attempt, and Recommend)便会在文献语料库中自动寻找候选问题、尝试求解,并逐层过滤出少数结果供专家评审。在组合数学领域的试点中,流程从5,245篇论文出发,最终筛出77项交由作者团队评审,其中作者亲自核查的15项全部数学上正确。

METAL MEDIA 解读图

FAR流水线:从文献语料库到专家评审结果

证据状态已报告实测结果

  1. Find 寻找按研究方向给论文打标签,抽取未解决的陈述,并核查其是否仍未解决,构成含4,717个猜想的可尝试问题池
  2. Attempt 尝试流水线中最强的模型对问题池中每个猜想各尝试一次,产生1,050个被标记为KNOWN/NEW/FIX/NONE之一的结果
  3. Recommend 推荐多位评判者核查NEW结果的正确性(598个通过),再由分级智能体将通过项分为已知、次要或可独立发表,最终保留77项
  4. 专家评审作者团队人工核查其中15项感兴趣的成果,全部确认正确,涉及多个知名猜想与问题
这是 METAL MEDIA 制作的解读图，并非论文作者提供的原图。

他们做了什么

  1. 在当前多数AI辅助数学研究的工作流中,人的精力集中在流程的开头(挑选合适的问题)和结尾(审阅AI产出的结果)这两个阶段,作者认为这正在成为高水平数学研究的瓶颈。
  2. 作者提出一种新范式:数学家不再事先选定单一问题,而只需给出一个自己感兴趣且擅长的研究方向,系统随后在广泛的文献语料库中搜索该方向下的候选问题,这一思路借鉴了搜索与推荐系统。
  3. 他们构建了FAR(Find, Attempt, and Recommend)流水线,依次完成:给论文打方向标签、从论文中抽取未解决的陈述、核查这些陈述是否仍然悬而未决(构成可尝试问题池),再用较强模型逐一尝试求解,最后对结果进行评判和推荐,交给专家评审。
  4. 在组合数学试点中,流程从51,110篇数学论文出发,标签阶段保留5,245篇组合数学论文,抽取阶段得到6,453个候选猜想或未解决问题,核查后剩4,717个看似表述良好且仍未解决的猜想构成问题池;后续尝试产生1,050个声称已解决的结果,评判阶段接受598个,分级阶段最终选出77项交由作者团队评审。
  5. 作者依据自身兴趣人工核查了其中15项,全部在数学上正确,涉及Davies-Jenssen-Perkins-Roberts、Erdős-Straus、Ikenmeyer-Pak-Panova以及Lund-Saraf-Wolf等人提出的猜想或问题。
Figure 1: From choosing a problem to choosing a direction. The upper part shows the problem-level interface; the lower part shows our approach. Figure 3 gives the details of FAR.
Figure 1: From choosing a problem to choosing a direction. The upper part shows the problem-level interface; the lower part shows our approach. Figure 3 gives the details of FAR.
Table 1: A recovered candidate, from source text to the pool.
FieldValue
Source paperIkenmeyer, Pak, and Panova, Positivity of the Symmetric Group Characters is as Hard as the Polynomial Time Hierarchy
Extracted labelConjecture 5.3.2
Extracted statementThe problem ComputeCharBinary is 𝖦𝖺𝗉𝖯-complete under many-one reductions.
StatusOpen. The check found no credible resolution, and records that the completeness question is still unsettled.
Figure 2: Schematic view of the current reachable region. The figure is illustrative, and both axes should be read qualitatively. A single pass over the pool probes which conjectures can produce artifacts worth review under the current model. As model capability improves, more conjectures may become reachable.
Figure 2: Schematic view of the current reachable region. The figure is illustrative, and both axes should be read qualitatively. A single pass over the pool probes which conjectures can produce artifacts worth review under the current model. As model capability improves, more conjectures may become reachable.
Table 2: Expected number of artifacts, the objective f1.
strategyB=10B=25B=50B=75B=100B=200B=300
uniform random0.170.430.861.291.723.445.17
rank on p^0.400.971.642.292.955.427.92
rank on i​p^0.150.471.021.592.244.787.25
rank on p^ inside the top 1/100.300.700.991.011.061.992.00
rank on p^ inside the top 1/50.170.420.861.251.682.993.72
Figure 3: From papers to recommendations for expert review. The upper row shows a search or recommender pipeline that recalls and filters candidates from a large corpus. Its numbers indicate typical orders of magnitude. The lower row shows the analogous FAR pipeline. Numbers in the lower row are counts from our pilot run detailed in Section 4.
Figure 3: From papers to recommendations for expert review. The upper row shows a search or recommender pipeline that recalls and filters candidates from a large corpus. Its numbers indicate typical orders of magnitude. The lower row shows the analogous FAR pipeline. Numbers in the lower row are counts from our pilot run detailed in Section 4.
Table 3: Expected total importance of the artifacts returned, the objective f2.
strategyB=10B=25B=50B=75B=100B=200B=300
uniform random0.080.210.420.630.841.682.51
rank on p^0.160.400.731.051.382.623.87
rank on i​p^0.090.280.600.891.222.493.72
rank on p^ inside the top 1/100.260.600.840.860.901.691.70
rank on p^ inside the top 1/50.130.340.691.001.342.443.06
Figure 4: Each score against the quantity it judges. Panel (a) plots δ(d−1[a,b)) on an axis starting at 50%, panel (b) plots ι(i−1[a,b)). n counts attempts in (a) and accepted resolutions in (b). Candidates that a later status recheck reclassified as solved or invalid are excluded.
Figure 4: Each score against the quantity it judges. Panel (a) plots δ(d−1[a,b)) on an axis starting at 50%, panel (b) plots ι(i−1[a,b)). n counts attempts in (a) and accepted resolutions in (b). Candidates that a later status recheck reclassified as solved or invalid are excluded.
Table 4: Expected maximum importance among the artifacts returned, the objective f3.
strategyB=10B=25B=50B=75B=100B=200B=300
uniform random0.0780.1760.2990.3860.4480.5740.629
rank on p^0.1570.3540.4370.4670.4850.4990.500
rank on i​p^0.0900.2800.5460.5970.6000.6000.600
rank on p^ inside the top 1/100.2590.5960.8410.8500.8500.8500.850
rank on p^ inside the top 1/50.1260.2940.5400.6900.7790.8500.850
The Problem Is the Problem: Towards Scalable Mathematical Discovery figure 4
Table 5: The base-Q digits of the parts of μ and of the two targets s and s−1. Here 𝟏 and 𝟎 denote the all-ones and all-zeros patterns on the block indicated, and 𝟏S denotes the indicator pattern of S on the X-block, with 𝟏S′ defined analogously on the X′-block. No column can reach Q, so subset sums may be compared digit by digit: a subset of parts summing to s must take cA, hence miss the X′-block entirely and pick out an exact cover of (X,C), while a subset summing to s−1 must take cB and pick out an exact cover of (X′,C′).
units digitX-blockX′-blockselector digit
Q0Q1,…,QkQk+1,…,Qk+k′Qk+k′+1
aS (S∈C)0𝟏S𝟎0
bS′ (S′∈C′)0𝟎𝟏S′0
cA1𝟎𝟏1
cB0𝟏𝟎1
H2𝟏𝟏1
s1𝟏𝟏1
s−10𝟏𝟏1
Figure 5: Allocation strategies against the budget. Each fit is made on four fifths of 𝒫 and applied to the remaining fifth, from which B/5 conjectures are drawn. The five selections together make one set of B conjectures, and each point averages what that set returns over 1000 random partitions. Ties are broken at random, and the uniform baseline is computed exactly from its closed form. Here, B only counts the allocated attempts.
Figure 5: Allocation strategies against the budget. Each fit is made on four fifths of 𝒫 and applied to the remaining fifth, from which B/5 conjectures are drawn. The five selections together make one set of B conjectures, and each point averages what that set returns over 1000 random partitions. Ties are broken at random, and the uniform baseline is computed exactly from its closed form. Here, B only counts the allocated attempts.

研究结果

  • 在组合数学试点中,51,110篇数学论文经标签筛选后剩5,245篇,抽取阶段得到6,453个候选猜想或未解决问题,核查后剩4,717个仍未解决、构成可尝试问题池。
  • 对问题池中每个猜想各尝试一次,共产生1,050个声称已解决的结果,评判阶段接受了598个,分级阶段最终留下77项作为交由作者团队评审的成果。
  • 作者依兴趣人工核查的15项成果全部在数学上正确,包括对Davies-Jenssen-Perkins-Roberts猜想和Lund-Saraf-Wolf猜想的反例、对Ikenmeyer-Pak-Panova猜想的证明,以及对Erdős-Straus提出问题的回答。
  • 模型给出的难度分数对'该问题最终未获接受结果'的预测AUC为0.69,重要性分数对'被接受结果是否评为可发表'的预测AUC为0.60,二者的相关性在统计上均显著(p分别小于10^-40和为0.008)。
  • 在比较预算分配策略时,按估计的成功概率(或成功概率乘以重要性)排序分配的策略,相比均匀分配基线能产生更多可发表成果,且最优策略会随目标(成果数量、总重要性还是单个最高重要性)不同而改变。
Figure 6: Each score against the quantity it judges, with Wilson intervals. Panel (a) gives δ against the difficulty score, panel (b) gives ι against the importance score.
Figure 6: Each score against the quantity it judges, with Wilson intervals. Panel (a) gives δ against the difficulty score, panel (b) gives ι against the importance score.

可应用场景

  • 在某一特定数学领域(如组合数学)中,自动扫描大量文献以发现仍未解决、值得尝试的猜想或问题
  • 在专家评审资源有限的场景下,用多阶段自动化审核流程对AI生成的大量候选证明或反例结果进行预筛选,减轻专家负担
  • 为如何在多个候选问题之间分配有限的模型推理预算提供参考,依据目标是追求更多成果数量、更高总重要性还是单个高影响力结果来选择分配策略

局限与待验证事项

  • 试点仅在组合数学这一作者本身具备核查能力的领域进行,尚未验证该方法在其他数学领域的适用性。
  • 实际运行中每个猜想只分配了一次尝试(相当于老虎机算法的初始化阶段),更复杂、允许重复尝试的分配算法留待未来研究。
  • 难度分数和重要性分数均由同一模型给出,二者的斯皮尔曼等级相关系数高达0.83,并非完全独立的信号。
  • 流水线自身对'是否已被解决'的核查并不完美,试点中就有一项结果其实在运行前几个月已由其他途径被解决,而级联中的各阶段搜索都未发现这一点。
  • 文中给出的最优预算分配策略是在假定预先知道每个问题成功概率和重要性的理论前提下推导的,论文并未报告实际估计误差对分配效果的具体影响。

为什么重要

前沿模型的推理能力和专家的数学评审能力都是稀缺资源,如何合理分配这两种资源直接决定了AI辅助数学研究能否真正提高效率。这项工作实际演示了一种把人力瓶颈从逐一挑选和审阅问题,转移到只需审阅少数经过多层筛选的高质量候选结果的具体做法。

本文术语

  • FAR(Find, Attempt, and Recommend) · 一个从文献中寻找问题、尝试求解、再推荐评审结果的三阶段流水线
  • 可尝试问题池(𝒫) · 从文献中抽取并核实为仍未解决的、有据可查的候选猜想集合
  • 推荐级联(recommendation cascade) · 搜索或推荐系统中常见的做法:用一系列逐渐更严格、更昂贵的过滤步骤,把庞大的候选集缩减到很小的一批
  • 多臂老虎机(bandit)视角 · 把有限的求解尝试次数如何分配给多个候选问题,看作多臂老虎机式的资源分配问题
  • AUC(ROC曲线下面积) · 衡量一个打分能否把结果更好的项目排在前面的指标,即随机抽一好一坏两项、好的排名更高的概率

论文原文摘要（英文）

AI systems are increasingly capable of contributing to mathematical research. In research practice, frontier-model reasoning is a limited resource, and expert mathematical review is even more sharply constrained. Allocating these scarce resources well is therefore central to making AI-assisted mathematical discovery efficient. In most current AI-for-math workflows, human effort is concentrated at the beginning and end, in selecting suitable research problems and later reviewing the resulting artifacts. These two stages are becoming bottlenecks for research-level mathematics. We address them by proposing a new human-AI discovery paradigm. The human input is no longer a single problem selected in advance, but a research direction in which the experts have interest and expertise. The system then searches a broad literature corpus for candidate problems in that direction. Inspired by search and recommender systems, we build Find, Attempt, and Recommend (FAR), a literature-to-review cascade that automates the search for suitable problems and focuses human attention on artifacts that have passed several stages of filtering. In a combinatorics pilot, the pipeline starts from 5,245 combinatorics papers, recovers 6,453 candidate conjectures or open problems, and filters them to 4,717 apparently well-posed and still-open conjectures. Subsequent reasoning and automated triage stages surface 598 potential resolutions and select 77 items for author-team review. Among them, we identify many interesting discoveries, including results on conjectures and questions of Davies--Jenssen--Perkins--Roberts, Erdős--Straus, Ikenmeyer--Pak--Panova, and Lund--Saraf--Wolf. These results demonstrate the effectiveness of this new mode of human-AI collaboration for mathematical discovery.

作者 · Zeyu Zheng

在 arXiv 阅读

最新论文

全部论文 →

METAL MEDIA 最新报道

图片来源： Zeyu Zheng et al., arXiv:2608.16977, cc-by-nc-sa-4.0