EnvHarness: Awakening Static Worlds for Agent Learning

arXiv:2608.198802026-08-21

Instead of building new training worlds from scratch, this work adds a plug-in layer that reshapes existing ones around each agent's actual weaknesses

LLM agents learn by practicing in environments, but hand-built environments stay frozen and eventually stop challenging an improving agent. EnvHarness wraps an existing environment with small plug-in components (changing the starting state, the allowed actions/observations, or chaining tasks together) without touching the environment's own logic or its trusted success-checker. An automated system called EnvRigger watches an agent's successes and failures, writes the right plug-in components to target its specific flaws, and tests them before keeping them.

METAL MEDIA explanatory visual

Instead of building new training worlds from scratch, this work adds a plug-in layer that reshapes existing ones around each agent's actual weaknesses

01Problem: existing agent-training environments are hand-built and static, so they can't target a specific agent's weaknesses and stop teaching once the agent masters them; prior auto-generation methods are domain-specific and depend on unreliable LLM-generated checkers.
02Solution: EnvHarness borrows the idea of an 'agent harness' (tools/memory added around a frozen model) and applies it to the environment side — a frozen environment gets wrapped by three component types: Stage (changes the starting state), Contract (changes allowed actions/observations/feedback), and Chain (links two environments into one longer task), all while keeping the original verifier intact.
03Automation: EnvRigger treats the target agent as a black box, observes its trajectories, diagnoses what's going wrong (e.g., repetitive loops, misreading long text, exploiting a shortcut), writes candidate components, and validates them with fresh rollouts before accepting them, revising or rejecting ones that don't work.
04Results: tested on five benchmarks in four domains (ALFWorld, WebArena, SWE-bench Verified, OfficeQA, SpreadsheetBench), agents trained with EnvHarness-reshaped environments beat both training on the original environments and domain-specific environment-generation baselines, gaining up to 9.0 points on held-out tasks while using 9.8% fewer execution steps on SWE-bench Verified.
05Extra findings: the same approach also produces a stronger reinforcement-learning training signal, keeps improving as more environments are added (while original/generated environments plateau), and works across different model backbones from weak to strong.

An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.

What they did

Problem: existing agent-training environments are hand-built and static, so they can't target a specific agent's weaknesses and stop teaching once the agent masters them; prior auto-generation methods are domain-specific and depend on unreliable LLM-generated checkers.
Solution: EnvHarness borrows the idea of an 'agent harness' (tools/memory added around a frozen model) and applies it to the environment side — a frozen environment gets wrapped by three component types: Stage (changes the starting state), Contract (changes allowed actions/observations/feedback), and Chain (links two environments into one longer task), all while keeping the original verifier intact.
Automation: EnvRigger treats the target agent as a black box, observes its trajectories, diagnoses what's going wrong (e.g., repetitive loops, misreading long text, exploiting a shortcut), writes candidate components, and validates them with fresh rollouts before accepting them, revising or rejecting ones that don't work.
Results: tested on five benchmarks in four domains (ALFWorld, WebArena, SWE-bench Verified, OfficeQA, SpreadsheetBench), agents trained with EnvHarness-reshaped environments beat both training on the original environments and domain-specific environment-generation baselines, gaining up to 9.0 points on held-out tasks while using 9.8% fewer execution steps on SWE-bench Verified.
Extra findings: the same approach also produces a stronger reinforcement-learning training signal, keeps improving as more environments are added (while original/generated environments plateau), and works across different model backbones from weak to strong.

EnvHarness: Awakening Static Worlds for Agent Learning figure 0

EnvHarness: Awakening Static Worlds for Agent Learning figure 1

EnvHarness: Awakening Static Worlds for Agent Learning figure 2

Table 7 | Training and evaluation splits per benchmark.
Benchmark	Training	Evaluation
ALFWorld	100 tasks from the standard train set	all remaining held-out tasks
WebArena	20 tasks per sub-domain	all remaining tasks
SWE-bench	100 tasks from SWE-bench Lite	407 Verified issues not in Lite
OfficeQA	50 tasks (official split)	172 official test tasks
SpreadsheetBench	100 of the 400 verified tasks	299 held-out tasks (897 instances)

Figure 2 | While an agent harness transforms a frozen LLM into a capable agent via plug-in components (e.g., skills, memory, tools) without altering model weights, EnvHarness applies this same principle to the other side of the interaction. It customizes a frozen environment with plug-in components while leaving original environment unchanged.

Figure 3 | Overview of EnvHarness components wrapping the standard environment inter- face. The underlying base environment (native state transitions and original task verifier) remains completely frozen. From left to right: the base environment, followed by three EnvHarness compo- nents—Stage, Contract, and Chain. Highlighted arrows and headers indicate overridden interface methods, with code blo

EnvHarness: Awakening Static Worlds for Agent Learning figure 5

Why it matters

For anyone training AI agents, this offers a way to keep a training environment useful over time without manually redesigning it for every new agent version. It suggests a general recipe — diagnose an agent's specific flaws, then reshape its practice ground around them — that could apply across coding, web, office, and embodied-task settings using the same underlying interface.

EnvHarness: Awakening Static Worlds for Agent Learning figure 6

EnvHarness: Awakening Static Worlds for Agent Learning figure 7

Figure 4 | EnvRigger generating EnvHarness components for a target policy based on given task. The execution loop on the left runs the policy against the current environment, which is a frozen base environment wrapped by the active EnvHarness containing accepted components 𝑤1, . . . , 𝑤𝑘, while the resulting rollout trajectories feed the EnvRigger loop on the right. The EnvRigger operates syste

Terms in this paper

LLM agent · An AI system built on a large language model that takes actions in an environment (e.g., clicking web pages, editing code) rather than just answering questions.
환경(environment)/verifier · The simulated task setting an agent interacts with, plus the checker that scores whether the agent succeeded.
Stage/Contract/Chain · The three plug-in component types: Stage changes the starting situation, Contract changes what actions/observations are allowed, Chain links two tasks into one longer episode.
black-box observation · Studying an agent only by watching its inputs/outputs and behavior, without looking inside its internal weights.
reinforcement learning (RL) / GRPO · A training method where a policy improves from reward signals collected by trying actions; GRPO is one such optimization algorithm used here.

Original abstract (English)

LLM agents learn by interacting with environments, yet these environments are hand-built and static: blind to an agent's weaknesses, and quickly left behind as it improves. While recent environment generation methods attempt to address this, they require domain-specific pipelines, rely on expensive or unreliable verifiers, and still produce static environments. To alleviate the engineering burden of rebuilding environments from scratch, we propose Environment Harness (EnvHarness), a programmable layer of plug-in components that wraps a static environment to reshape its behavior without modifying the underlying logic. Operating through standard interfaces, EnvHarness applies across diverse domains while ensuring every reshaped environment retains its original verifier. To automate this process, we introduce EnvRigger, which treats the target policy as a black box, observing its execution trajectories to synthesize EnvHarness components targeting diagnosed flaws, and validating them via fresh rollouts. Across five benchmarks in four domains, EnvHarness outperforms both original environments and domain-specific environment generation pipelines, achieving up to a 9.0-point improvement on held-out instances with 9.8% fewer execution steps. Furthermore, EnvHarness provides a superior optimization signal for reinforcement learning, enabling continuous, targeted co-evolution of the policy and its environment.

Authors · Chengsong Huang, Zifeng Wang, Rujun Han, Jun Yan, Yanfei Chen, Zoey CuiZhu, Ke Jiang, Peng Xia, Han Yu, Yufan Zhuang, Yi

Read on arXiv

Latest papers

All papers →

Latest from METAL MEDIA

NCT 127 drop BLINGY, their first full album in two years

Figures: Chengsong Huang et al., arXiv:2608.19880, CC BY 4.0