DART-SD: Diamond-topology Aware Retrieval and Tuning for Self-Distillation of Multi-Turn Tool-Calling Agents

What they did

1. Problem: tasks with order-independent sub-goals form a diamond-shaped lattice of valid solution paths, but standard SFT and GRPO-style reinforcement learning flatten this into a single linear trajectory, wrongly penalizing valid alternative paths the student model finds on its own
2. Method: teacher rollouts (both successful and failed) are turned into a graph called the Interaction-State Transition Graph (ISTG); a student's failed attempt is mapped onto this graph to find the earliest point where it left the region still reachable to success, called the Critical Topological Breakpoint (CTB), and training loss is applied only to the recovery continuation generated after that point, repeated over multiple rounds
3. Evaluation: trained on the FTRL dataset and tested on five benchmarks (FTRL, BFCL, ToolHop, tau-bench, RoTBench), DART-SD outperformed existing distillation methods (SCoRe-SFT, OPSD) and reinforcement learning methods (FTRL-GRPO, ToolRL, MatchTIR) on both Qwen3-4B and Qwen3-8B backbones
4. Efficiency gains: across training iterations, average tool calls in successful trajectories dropped from 4.23 to 3.55 while task success kept improving, ending up shorter than the human-constructed reference trajectories (average 4.02 calls); the point of first failure (CTB) also shifted later over iterations, showing the student's reliable range kept extending
5. General capability tests (instruction following, math reasoning, general knowledge) improved on average from 43.92 to 49.89, showing that tool-use training didn't come at the cost of the model's other abilities

Why it matters

It shows that for AI agents that repeatedly call tools to solve problems, forcing them to memorize one fixed correct sequence is wasteful and can destroy useful strategies the model already has. Precisely locating and fixing only the failure point offers a more efficient way to build smaller, smarter, and more efficient tool-using agents.

Terms in this paper

• SFT (Supervised Fine-Tuning) · Additional training where a model is taught to copy given example answers
• GRPO · A reinforcement learning method that adjusts a model's behavior using reward signals
• ISTG (Interaction-State Transition Graph) · A graph representing tool-use progress as nodes based on what information has been acquired
• CTB (Critical Topological Breakpoint) · The first point where a student's attempt departs from a path that could still succeed
• Self-distillation · Retraining a model using outputs it generated itself

Figures we cannot republish

• Figure 1: Comparison of training paradigms. (a) SFT Teacher Boosting applies an indiscriminate global loss, which overwrites valid exploration. (b) Standard-RL (GRPO) misassigns credit through a uniformly distributed reward spread. (c) DART-SD dynamically identifies the Critical Topological Breakpoints (CTB) and applies localized correction while preserving valid exploration.
• Figure 2: Overview of DART-SD. (1) DART-SD constructs an ISTG from teacher rollouts, where main and auxiliary nodes model information acquisition and useless exploration. (2–3) Failed student rollouts are collected and replayed in the same interaction-state space, then projected onto the budget-filtered success-reachable region ℛx+, where the first projectable-to-non-projectable transition defines the CTB. (4–5) Conditioned on the retained student prefix and privileged teacher references, DART-SD generates a recovery continuation after the CTB and applies localized supervision only to the generated assistant tokens. (6) This CTB-guided SFT loop is progressively repeated across self-distillation rounds.
• Figure 3: Performance comparison of Qwen3-8B, DART-SD, and the teacher across five tool-use benchmarks. DART-SD improves upon Qwen3-8B on all benchmarks and surpasses the teacher on FTRL, ToolHop, and τ-bench.

Latest papers

• SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?AI coding agents were tested on fixing real scientific software, and even the best one failed more than half the timearXiv:2608.197992026-08-21
• FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM ServingMaking sparse attention fast enough and accurate enough for real LLM serving, not just papersarXiv:2608.197582026-08-21
• PolicyGuide: From Guarding One Action to Guiding the Whole Workflow for Policy-Compliant LLM AgentsMaking customer-service AI agents follow the whole procedure, not just avoid one bad actionarXiv:2608.198612026-08-21
• EXIMO: VLM Guided Exploration of VLA PoliciesTeaching a robot new chores without human teleoperation, by letting a chatty AI supervise itarXiv:2608.198912026-08-21
• EnvHarness: Awakening Static Worlds for Agent LearningInstead of building new training worlds from scratch, this work adds a plug-in layer that reshapes existing ones around each agent's actual weaknessesarXiv:2608.198802026-08-21
• Bounded Sovereignty and the Control Tax: Pricing AI Oversight When the Deployer Does Not Own the ModelCompanies that rent AI instead of owning it can only do half of AI safety oversightarXiv:2608.192162026-08-21
• Beyond Imitation: Filtering On-Policy Distillation by Reasoning ProgressA fix for AI models that get penalized by their teacher even when they're reasoning correctlyarXiv:2608.194082026-08-21
• PersonalBench: Measuring the Authorship Gap in LLM PersonalizationAI can be prompted to write 'like someone,' but its own voice never fully disappearsarXiv:2608.197462026-08-21

Latest from METAL MEDIA

• TUIDE debut with 'SUN KISS,' an Iceland-shot K-pop road movie
• NCT 127 drop BLINGY, their first full album in two years