Compiler-Guided Adaptive Proof Search with Cross-Model Synergy on Context-Dependent Theorem Proving

What they did

1. Two complementary models generate candidate proofs: a general-purpose reasoning model (generalist) and a Lean-specialized prover (specialist); an LLM judge picks the more promising one as the starting point.
2. The chosen proof is repeatedly revised using compiler error feedback, but each revision is only accepted if the judge decides it's actually better than the current best; otherwise the old version is kept.
3. When repair stalls for several rounds without improvement (a stagnation counter), the system resamples fresh candidates from both models and restarts refinement.
4. On miniCTX-v2's seven projects, within a pass@32 budget the method improved average pass rate by 12.8 percentage points and cut LLM calls by 21.9%, at lower cost ($1.96 vs $2.43 for GPT-5-mini pass@32).
5. Analysis showed about half (50.4%) of proposed repairs get rejected by the judge, and 39.5% of successful proofs came from a preserved earlier state rather than the newest revision, showing that keeping good states matters as much as generating new ones.

Why it matters

Real-world, project-scale theorem proving is hard because it depends heavily on local context, and this work shows that smarter reuse of failed attempts—not just bigger or retrained models—can improve both accuracy and cost efficiency. The exploration-versus-exploitation search strategy here could generalize to other AI tasks that receive compiler or execution feedback, such as code repair or agentic tool use.

Terms in this paper

• Lean 4 · A programming language and proof assistant used to write and machine-check formal mathematical proofs
• pass@k · An evaluation metric counting success if any of k sampled attempts verifies
• compiler feedback · Error messages from the Lean compiler indicating where and why a proof attempt failed
• pairwise comparison (LLM judge) · An LLM prompted to compare two candidate proofs and pick the one more likely to succeed
• stagnation threshold (N) · The number of consecutive rejected refinements after which the system restarts exploration with fresh candidates

Figures we cannot republish

• Figure 1: Overview of our framework. Exploration (left) generates two diverse candidates via a generalist and a specialist. Exploitation (right) maintains a single current-best proof s∗ and drives it toward verification: each repair produces a proposal, and an LLM judge decides whether to accept it. When refinement stagnates for N consecutive rounds, the system re-enters exploration and draws fresh candidates from both models. The Verification & Feedback layer (bottom) provides structured error feedback, tactic suggestions and verification signals.
• Figure 2: Empirical refinement results of one theorem with different initial attempts. One theorem, 10 wrong proofs (WP) as starting points (rows), each iteratively refined 5 times (columns) with a 32-iteration budget. Cells show iterations to success; × marks failure. Theorem: Set_EAnnulus_oc_subset_co in carleson. More examples can be found in Figure 8 and Figure 9 in Appendix B.
• Figure 3: Pass rate (%) vs. LLM calls across seven Lean 4 projects from miniCTX-v2. Gray dashed line is pass@32.
• Figure 4: LLM cost breakdown under 32 average LLM calls. Specialist DeepSeek-Prover-V2-7B is self-hosted; its cost is estimated at the generalist’s per-token price (higher than the specialist) for a conservative comparison.
• Figure 5: Pass rate (%) vs. LLM calls on RLMEval-FLT3 with informal proof guidance. Text labels show average calls from the generalist, specialist, and pairwise comparison (Cmp).
• Figure 6: Ablation study results.
• (b) Effect of pairwise comparison on call allocation and cost.
• Figure 7: Pairwise comparison behavior on solved refinement trajectories. (a) Keep-current rate during successful refinement. (b) Final verification from preserved proof states.
• Figure 8: Refinement heatmap for _isBigO_deriv_ofReal_cpow_const_atTop in the mathlib.
• Figure 9: Refinement heatmap for eLpNorm_toReal_le in the Carleson.
• Figure 10: LLM calls breakdown under 32 average LLM calls on miniCTX-v2 test.
• Figure 11: Pass rate (%) vs. LLM calls on HepLean with GPT5.4-mini as the Generalist and Goedel-Prover-V2-8B as the Specialist.
• Figure 12: Global refinement keep current rate on solved theorems (294 total steps).

Latest papers

• SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?AI coding agents were tested on fixing real scientific software, and even the best one failed more than half the timearXiv:2608.197992026-08-21
• FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM ServingMaking sparse attention fast enough and accurate enough for real LLM serving, not just papersarXiv:2608.197582026-08-21
• PolicyGuide: From Guarding One Action to Guiding the Whole Workflow for Policy-Compliant LLM AgentsMaking customer-service AI agents follow the whole procedure, not just avoid one bad actionarXiv:2608.198612026-08-21
• EXIMO: VLM Guided Exploration of VLA PoliciesTeaching a robot new chores without human teleoperation, by letting a chatty AI supervise itarXiv:2608.198912026-08-21
• EnvHarness: Awakening Static Worlds for Agent LearningInstead of building new training worlds from scratch, this work adds a plug-in layer that reshapes existing ones around each agent's actual weaknessesarXiv:2608.198802026-08-21
• Bounded Sovereignty and the Control Tax: Pricing AI Oversight When the Deployer Does Not Own the ModelCompanies that rent AI instead of owning it can only do half of AI safety oversightarXiv:2608.192162026-08-21
• Beyond Imitation: Filtering On-Policy Distillation by Reasoning ProgressA fix for AI models that get penalized by their teacher even when they're reasoning correctlyarXiv:2608.194082026-08-21
• PersonalBench: Measuring the Authorship Gap in LLM PersonalizationAI can be prompted to write 'like someone,' but its own voice never fully disappearsarXiv:2608.197462026-08-21

Latest from METAL MEDIA

• TUIDE debut with 'SUN KISS,' an Iceland-shot K-pop road movie
• NCT 127 drop BLINGY, their first full album in two years