Looped Language Models Improve Compositional Tool Calling

arXiv:2608.181712026-08-20

AI models that rethink their own answers multiple times get better at chaining tools together

Researchers tested looped language models, which pass their internal computation through the same block repeatedly before answering, on three tool-calling benchmarks: API-Bank, BFCL, and NESTful. Comparing looped and non-looped models trained with identical data and settings, looped models showed clear gains on tasks requiring coordinating multiple API calls and passing outputs between them, while gains on single, isolated calls were smaller and inconsistent. An adaptive inference method that only adds extra computation when needed matched or beat fixed-depth looping while using less compute.

METAL MEDIA explanatory visual

AI models that rethink their own answers multiple times get better at chaining tools together

01Tested both natively looped models (Ouro-1.4B, Ouro-2.6B) and models retrofitted into looped architectures from Llama-3.2-1B and OLMo-2-1B
02Compared looped vs. non-looped models trained on the same Hermes function-calling dataset with matched settings
03On single-call tasks the benefit of looping was small and model-dependent, but on tasks requiring multiple parallel calls or dependent calls (where one call's output feeds another), the gains were much larger
04Varying recurrent depth from 1 to 8 iterations showed accuracy on compositional tasks generally rising with more iterations before plateauing
05Adaptive inference, which lets the model exit the loop per-token once confident, achieved similar or better results than fixed-depth looping while using fewer iterations on average; examples showed models correcting missing dependent calls and wrong function choices as depth increased

An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.

What they did

Tested both natively looped models (Ouro-1.4B, Ouro-2.6B) and models retrofitted into looped architectures from Llama-3.2-1B and OLMo-2-1B
Compared looped vs. non-looped models trained on the same Hermes function-calling dataset with matched settings
On single-call tasks the benefit of looping was small and model-dependent, but on tasks requiring multiple parallel calls or dependent calls (where one call's output feeds another), the gains were much larger
Varying recurrent depth from 1 to 8 iterations showed accuracy on compositional tasks generally rising with more iterations before plateauing
Adaptive inference, which lets the model exit the loop per-token once confident, achieved similar or better results than fixed-depth looping while using fewer iterations on average; examples showed models correcting missing dependent calls and wrong function choices as depth increased

Figure 2: BFCL semantic AST accuracy by task category as fixed recurrent depth increases. Each curve reports accuracy on Simple, Multiple, Parallel, and Parallel-Multiple tasks.

Figure 3: NESTful Win Rate as fixed recurrent depth increases. Native recurrent Ouro models benefit from additional recurrent computation, whereas the retrofitted recurrent Llama baseline remains substantially weaker.

Table 1: BFCL semantic AST correctness (%; higher is better). Overall is the aggregate score across all task categories. The lower block compares looped and non-looped variants of each backbone, fine-tuned using identical data and optimization settings; looped variants use a fixed inference depth of 8 loop iterations. Base checkpoints omitted from the lower block score near zero throughout.
Model	Training	Simple	Multiple	Parallel	Par.-Mult.	Overall
Ouro-1.4B	Base	65.3	67.0	35.0	44.5	55.4
	SFT	91.8	90.0	67.5	55.5	79.3
Ouro-2.6B	Base	80.5	75.5	2.0	1.0	47.9
	SFT	92.3	88.0	83.0	76.5	86.4
Qwen3-1.7B	Base	1.5	0.0	0.0	0.0	0.6
	SFT	67.0	57.0	3.0	9.5	40.7
	Instruct	91.8	91.5	83.5	81.0	87.9
Qwen3-4B	Base	72.0	57.5	1.0	0.5	40.6
	SFT	94.3	87.0	2.5	5.5	56.7
	Instruct	93.0	92.5	87.5	88.5	90.9
Qwen3-8B	Instruct	95.5	96.0	91.5	89.5	93.6
Llama-3.2-1B	Instruct	18.2	2.5	4.0	4.5	9.5
Llama-3.2-3B	SFT	88.2	86.0	69.0	60.5	78.4
	Instruct	33.8	37.0	0.0	0.0	20.9
Llama-3.1-8B	Instruct	46.5	41.0	0.0	0.0	26.8
OLMo-2-1B	SFT	59.2	50.0	14.5	12.5	39.1
	Looped SFT	55.0	58.0	26.5	14.5	41.8
Llama-3.2-1B	SFT	29.8	28.5	14.0	5.0	21.4
	Looped SFT	43.5	40.5	31.0	6.0	32.9

Figure 4: Adaptive recurrent computation on BFCL and NESTful. The x-axis reports the mean recurrent depth per generated token. Adaptive stopping improves the compute-performance frontier by allocating additional recurrent iterations only when beneficial. On BFCL, adaptive stopping matches or slightly exceeds the best fixed-depth operating point while executing fewer recurrent iterations on average. On NESTful, Ouro-2.6B reaches the same Win Rate as fixed depth 4 while using fewer recurrent iterations per generated token.

Table 2: NESTful official evaluation (higher is better). Win Rate is the primary metric.
Model	Training	Function F1	Parameter F1	Partial	Full	Win Rate
Ouro-1.4B	Base	0.905	0.539	0.149	0.091	0.110
	SFT	0.899	0.566	0.219	0.131	0.191
Ouro-2.6B	Base	0.920	0.595	0.207	0.128	0.190
	SFT	0.922	0.680	0.295	0.204	0.371
Qwen3-1.7B	Base	0.000	0.000	0.000	0.000	0.000
	SFT	0.000	0.000	0.000	0.000	0.000
	Instruct	0.924	0.555	0.202	0.109	0.134
Qwen3-4B	Base	0.000	0.000	0.000	0.000	0.000
	SFT	0.458	0.309	0.156	0.001	0.063
	Instruct	0.971	0.703	0.285	0.196	0.292
Qwen3-8B	Instruct	0.979	0.774	0.329	0.246	0.345
Llama-3.2-3B	Base	0.911	0.561	0.208	0.155	0.146
	SFT	0.911	0.495	0.175	0.093	0.095
	Instruct	0.929	0.419	0.160	0.033	0.060
Llama-3.1-8B	Instruct	0.657	0.342	0.137	0.030	0.073

Figure 5: Iterative refinement across recurrent depths. Ouro-1.4B on a two-step NESTful composition task. Calls are shown as function(argument=value) -> answer. At depth 1 the model emits a single call to a function absent from the tool catalogue and omits the dependent call; at depth 2 it recovers the two-call structure but the first function is again absent from the catalogue and the variable reference is invalid. Depths 3 and 4 match the gold sequence, including the output-to-input reference $var1.output_0$.

Table 3: API-Bank evaluation (%; higher is better). (a) Controlled comparison between looped and non-looped models trained using identical supervised fine-tuning recipes. The lower block pairs each retrofitted backbone with its non-looped counterpart; looped variants are evaluated at a fixed inference depth of 8 loop iterations. (b) Comparison against publicly released instruction-tuned checkpoints. Call Correctness measures functional API-call correctness, Exact requires an exact API-name-and-argument match, and Parse reports valid call generation.
(a) Controlled models
Base SFT Model Call Exact Parse Call Exact Parse Ouro-1.4B 73.0 67.6 95.1 75.1 70.2 97.2 Ouro-2.6B 79.2 76.9 99.2 79.9 77.1 99.7 Qwen3-1.7B 5.1 5.1 13.9 61.4 57.8 93.6 Qwen3-4B 71.2 71.2 99.2 76.6 73.8 99.7 Llama-3.2-1B 1.7 1.3 10.8 16.3 14.1 55.3 Llama-3.2-3B 0.3 0.3 0.3 68.5 64.5 99.7 OLMo-2-1B 1.9 0.5 35.7 37.1 33.2 99.5 OLMo-2-1B (Loop) 0.1 0.0 1.5 34.0 30.3 90.7 Llama-3.2-1B 1.7 1.3 10.8 16.3 14.1 55.3 Llama-3.2-1B (Loop) 0.1 0.0 1.5 17.9 16.2 43.4

Figure 6: Ordering and spurious-call correction. At depth 1 the output is malformed, emitting tool specifications rather than call arguments (excerpt shown truncated). At depth 2 both gold functions appear but in reversed order, followed by a call to multiply, which is absent from the tool catalogue, and an invalid variable reference. Depths 3 and 4 match the gold sequence, differing from each other only in label naming.

Figure 7: Argument correction. At depth 1 the model selects the correct first function but names its argument input_str, a parameter of a different tool in the same catalogue, duplicates it as input_str_2, and omits the dependent call. Depths 2 through 4 match the gold sequence and are identical to one another.

Table 4: Structural comparison of the evaluated tool-calling benchmarks under the solution representation Gx=(Cx,Ex). API-Bank emphasizes individual-call grounding, BFCL evaluates increasingly complex independent call sets, and NESTful introduces explicit output-to-input dependencies.
Benchmark	Candidate tools	Call structure	Dependencies	Primary challenge
API-Bank	one or more	typically \|Cx\|=1	Ex=∅	Tool and argument grounding
BFCL Simple	\|𝒯x\|=1	\|Cx\|=1	Ex=∅	Argument grounding
BFCL Multiple	\|𝒯x\|>1	\|Cx\|=1	Ex=∅	Function selection
BFCL Parallel	one or more	\|Cx\|>1	Ex=∅	Independent call generation
BFCL Parallel-Multiple	\|𝒯x\|>1	\|Cx\|>1	Ex=∅	Selection and call composition
NESTful	multiple	\|Cx\|>1	typically \|Ex\|>0	Dependency-aware execution

Figure 8: Two distinct incorrect attempts before recovery. Depths 1 and 2 each emit a single call to a function absent from the tool catalogue and omit the required dependent call. At depth 1, the model also uses an argument name belonging to a different tool in the catalogue. Depths 3 and 4 select both gold functions and correctly pass $var1.output_0$, differing only in argument-key order.

Figure 9: Isolated recovery of a missing dependent call. Depth 1 emits an object with no function name, returning a computed literal, so the parser yields no calls. Depth 2 emits a first call that matches the gold sequence exactly but stops there. Depths 3 and 4 leave that first call unchanged and add the dependent call with the reference $var1.output_0$, isolating the recovery of the dependency from any change in function selection.

Why it matters

This matters for anyone building AI agents that must orchestrate multiple tools in sequence, since it suggests accuracy can be improved by spending more compute at inference time rather than scaling up model size. The adaptive approach also points to a practical way to control compute cost while still handling complex, dependency-heavy workflows.

Terms in this paper

Looped Language Model · a model that repeatedly applies the same neural network block before producing an answer, increasing computation without adding parameters
Recurrent Depth · the number of times the shared block is applied during inference
Adaptive Inference · a method where the model decides per-token how many loop iterations to run, balancing compute and accuracy
Retrofitted Model · a pretrained standard model converted into a looped architecture by sharing some of its layers across iterations
Semantic AST Accuracy · a scoring method that standardizes generated function calls into a structured form and checks if they semantically match the correct answer

Original abstract (English)

Looped language models have shown promising results on reasoning benchmarks, yet their potential for agentic tool use remains largely unexplored. We study this question in compositional tool-calling settings, where models must coordinate multiple API calls, maintain intermediate state, and preserve dependencies across tool interactions. We evaluate native and retrofitted looped language models on API-Bank, BFCL, and NESTful, comparing looped and non-looped models trained under matched supervised fine-tuning recipes and varying recurrent depth at inference time. In controlled experiments, recurrent computation generally benefits compositional and dependency-aware tool use, while providing smaller and more model-dependent gains on isolated API invocation. Accuracy on multi-step tool use generally increases with recurrent depth; adaptive inference, however, achieves a more favorable compute-performance trade-off by allocating additional computation only when needed. Our results suggest that looped language models are a promising architecture for agentic systems that require reliable planning, coordination, and execution of compositional tool use workflows.

Authors · Andrei Cristian Popescu, Haitz S\'aez de Oc\'ariz Borde, Pietro Li\`o

Read on arXiv

Latest papers

All papers →

Latest from METAL MEDIA

NCT 127 drop BLINGY, their first full album in two years

Figures: Andrei Cristian Popescu et al., arXiv:2608.18171, CC BY 4.0