K-文化的一切——从回归到 K-美妆，发送到您的邮箱订阅邮件

METAL MEDIA

Zetta ζ: An Efficient Closed-Loop Embodied Harness for Self-Evolving Physical Intelligence

arXiv:2608.165902026-08-16

Zetta让机器人边干活边发现并纠正自己的错误,还能越做越好

Zetta的做法是让控制机器人的核心AI模型保持不变,而是在外层不断进化一套实时监控代码和纠错动作组成的“监管层”,随着尝试次数增加成功率不断提升。系统会自动把失败的尝试归类、诊断原因,并转化成可重复使用的纠错技能,配合专门搭建的执行基础设施Z-Infra一起运作。最终在LIBERO-Pro上达到90.8%、在RoboCasa上达到93.6%的成功率,推理速度也提升了11.1倍。

METAL MEDIA 解读图

Zetta让机器人边干活边发现并纠正自己的错误,还能越做越好

  1. 01现有的具身智能体系统大多只在整个任务尝试结束后才做事后反思,无法应对机器人运动过程中实时发生的问题,比如物体滑落或碰撞。
  2. 02Zetta完全不改动控制机器人的底层AI模型(VLA/WAM),而是在其上叠加高频率运行的“评判器”监控代码和相应的“恢复”动作,实现实时监管。
  3. 03系统设置了三个不同时间尺度的循环:一个按动作频率实时监管,一个在多次尝试之后批量诊断并提出改进候选,还有一个只在验证通过后才把改进正式纳入记忆。
  4. 04失败案例会被自动按原因聚类,再按照评估、监控、状态识别、规划、恢复、参数这些层级由高到低依次排查,只在真正需要的最小层面上打补丁,避免过度修改导致其他情况失败。
  5. 05Z-Infra把智能体的逻辑和底层硬件(不同的GPU、CPU、仿真环境)解耦开,把有效尝试的处理速度从每分钟1.7次提升到35.1次,提升了20.6倍。
这是 METAL MEDIA 制作的解读图，并非论文作者提供的原图。

他们做了什么

  1. 现有的具身智能体系统大多只在整个任务尝试结束后才做事后反思,无法应对机器人运动过程中实时发生的问题,比如物体滑落或碰撞。
  2. Zetta完全不改动控制机器人的底层AI模型(VLA/WAM),而是在其上叠加高频率运行的“评判器”监控代码和相应的“恢复”动作,实现实时监管。
  3. 系统设置了三个不同时间尺度的循环:一个按动作频率实时监管,一个在多次尝试之后批量诊断并提出改进候选,还有一个只在验证通过后才把改进正式纳入记忆。
  4. 失败案例会被自动按原因聚类,再按照评估、监控、状态识别、规划、恢复、参数这些层级由高到低依次排查,只在真正需要的最小层面上打补丁,避免过度修改导致其他情况失败。
  5. Z-Infra把智能体的逻辑和底层硬件(不同的GPU、CPU、仿真环境)解耦开,把有效尝试的处理速度从每分钟1.7次提升到35.1次,提升了20.6倍。
Figure 1: Zetta closes the loop for embodied self-evolution. Frequent runtime critics trigger recoveries during execution, while verified failures are distilled into reusable critic and recovery skills across rollouts. A hardware-decoupled rollout layer scales this process across heterogeneous environments, models, CPUs, and GPUs. Zetta enables sustained same-task improvement, zero-shot skill transfer, robotic "Aha moment", and accelerated agent execution.
Figure 1: Zetta closes the loop for embodied self-evolution. Frequent runtime critics trigger recoveries during execution, while verified failures are distilled into reusable critic and recovery skills across rollouts. A hardware-decoupled rollout layer scales this process across heterogeneous environments, models, CPUs, and GPUs. Zetta enables sustained same-task improvement, zero-shot skill transfer, robotic "Aha moment", and accelerated agent execution.
Figure 2: Overview of the Zetta evolutionary framework. The system operates in a continuous loop between online execution and offline evolution. (Left) Parallel Rollouts: Action policy executes tasks, monitored by an Evolvable Harness (ℋ) composed of Critics (C), Recovery (R), and Tools (T), all under the high-level adjudication of an Orchestrator Agent. Rollouts are categorized into success and failure trajectories. (Right) Reflection & Evolve: Failed trajectories trigger a three-phase offline evolution cycle. Phase I profiles failures by clustering them against successful reference baselines. Phase II performs causal diagnosis to locate the root failure layer (L∗), followed by minimal harness repair to update C, R, and T. Phase III generalizes seed-specific repairs into a unified, versioned Harness Package (ℋm​e​r​g​e​d), which is then fed back into the execution loop.
Figure 2: Overview of the Zetta evolutionary framework. The system operates in a continuous loop between online execution and offline evolution. (Left) Parallel Rollouts: Action policy executes tasks, monitored by an Evolvable Harness (ℋ) composed of Critics (C), Recovery (R), and Tools (T), all under the high-level adjudication of an Orchestrator Agent. Rollouts are categorized into success and failure trajectories. (Right) Reflection & Evolve: Failed trajectories trigger a three-phase offline evolution cycle. Phase I profiles failures by clustering them against successful reference baselines. Phase II performs causal diagnosis to locate the root failure layer (L∗), followed by minimal harness repair to update C, R, and T. Phase III generalizes seed-specific repairs into a unified, versioned Harness Package (ℋm​e​r​g​e​d), which is then fed back into the execution loop.
Table 1: Core API primitives exposed to agents. All primitives support batching across sessions.
API PrimitiveFunctionality
Session Management — create once, run many episodes, close when done
create_sessions(requests)Batch create sessions with env_family, env_config, lease_seconds. Returns SessionHandle with session_id.
renew_sessions(session_ids)Extend lease duration for active sessions.
close_sessions(session_ids)Close sessions and release resources (idempotent).
Environment Control — direct interaction for custom control logic
reset(session_ids, reset_spec)Start new episode with task_id, seed, instruction. Returns episode_id and initial observation.
observe(session_ids)Read current observation without side effects.
action_step(session_ids, actions)Execute actions, return observation, reward, terminated, truncated flags.
Policy Inference — integrated model serving and stepping
policy_step(session_ids, policy_req)Atomic observe → inference → step. Returns step results with executed actions.
policy_infer(session_ids, policy_req)Inference only (no stepping). Returns actions for agent post-processing.
run_episode(session_ids, episode_req)Execute complete episodes in workers. Returns summary (steps, reward, stop_reason).
Figure 3: Three-layer architecture of the rollout infrastructure. The Control Plane routes agent requests to specialized Env Workers and Rollout Workers, which manage environment simulation and model inference respectively.
Figure 3: Three-layer architecture of the rollout infrastructure. The Control Plane routes agent requests to specialized Env Workers and Rollout Workers, which manage environment simulation and model inference respectively.
Figure 4: Physical-intelligence “Aha” moments on LIBERO-Pro. Similar to RoboCasa, v0 denotes the Pure-VLA baseline and v1 represents early symptomatic repairs (e.g., staging or local gates) that yield stagnant performance. The “Aha” at v2 occurs when the agent identifies and resolves the decisive physical bottleneck (e.g., grasp retention or semantic approach), triggering a sharp increase in execution reliability.
Figure 4: Physical-intelligence “Aha” moments on LIBERO-Pro. Similar to RoboCasa, v0 denotes the Pure-VLA baseline and v1 represents early symptomatic repairs (e.g., staging or local gates) that yield stagnant performance. The “Aha” at v2 occurs when the agent identifies and resolves the decisive physical bottleneck (e.g., grasp retention or semantic approach), triggering a sharp increase in execution reliability.
Table 2: Success rates (%) on 18 RoboCasa Atomic-Seen tasks. Task identifiers follow Appendix A. The best result for each task is shown in bold. “Avg.” is the macro-average over all 18 tasks.
MethodT1T2T3T4T5T6T7T8T9Avg.
Pure VLA (GR00T)78747858486272747073.56
Zetta96929496868096968693.56
MethodT10T11T12T13T14T15T16T17T18Avg.
Pure VLA (GR00T)62927688965090827473.56
Zetta9498949410010094969293.56
Figure 5: Physical-intelligence “Aha” moments on RoboCasa. L2-v0 is the original Pure-VLA baseline, while L2-v1 is an intermediate internal version. The flat L2-v0–L2-v1 segments summarize a period where early candidate fixes overfit individual failures. Once the agent identifies the key physical bottleneck (e.g., EEF alignment or centered contact), L2-v2 produces a sharp success-rate increase.
Figure 5: Physical-intelligence “Aha” moments on RoboCasa. L2-v0 is the original Pure-VLA baseline, while L2-v1 is an intermediate internal version. The flat L2-v0–L2-v1 segments summarize a period where early candidate fixes overfit individual failures. Once the agent identifies the key physical bottleneck (e.g., EEF alignment or centered contact), L2-v2 produces a sharp success-rate increase.
Figure 6: Cumulative physical-intelligence scaling on LIBERO-Pro Goal. Average performance across ten tasks under Goal-T and Goal-S perturbations. Each point denotes a selected cumulative harness version. The base VLA is frozen throughout evolution.
Figure 6: Cumulative physical-intelligence scaling on LIBERO-Pro Goal. Average performance across ten tasks under Goal-T and Goal-S perturbations. Each point denotes a selected cumulative harness version. The base VLA is frozen throughout evolution.
Table 3: Success rates (%) on LIBERO-Pro. The best result for each task is shown in bold. “Average” is the macro-average over 10 tasks.
SettingMethodTask 0Task 1Task 2Task 3Task 4Task 5Task 6Task 7Task 8Task 9Average
Goal (T)π0.50.095.010.00.0100.00.020.080.05.00.031.0
Zetta80.0100.095.080.0100.0100.095.095.080.0100.092.5
Goal (S)π0.50.060.00.045.00.00.00.0100.0100.075.038.0
Zetta90.065.080.085.095.095.0100.0100.0100.080.089.0
LIBERO-10 (T)π0.55.095.095.00.00.080.085.075.065.00.050.0
Zetta35.095.0100.00.025.0100.095.080.0100.00.063.0
LIBERO-10 (S)π0.50.035.00.00.05.050.00.00.00.00.09.0
Zetta90.050.095.075.015.065.05.00.00.05.040.0
(b) Swap/position-swap (S).
(b) Swap/position-swap (S).
Figure 7: Cross-task scaling from the Goal-T8 source task. Three cumulative Critic–Recovery capabilities discovered on Goal-T8 are transferred to Goal-T2, Goal-T6, and Goal-S3. Curves report complete fixed-seed evaluations of each cumulative stack. Shaded regions denote Wilson 95% confidence intervals. The right-hand frames illustrate the corresponding failure and recovery mechanisms rather than additional transfer measurements.
Figure 7: Cross-task scaling from the Goal-T8 source task. Three cumulative Critic–Recovery capabilities discovered on Goal-T8 are transferred to Goal-T2, Goal-T6, and Goal-S3. Curves report complete fixed-seed evaluations of each cumulative stack. Shaded regions denote Wilson 95% confidence intervals. The right-hand frames illustrate the corresponding failure and recovery mechanisms rather than additional transfer measurements.
Table 4: Mapping from compact identifiers to official RoboCasa task names.
IDOfficial task name
T1NavigateKitchen
T2TurnOnMicrowave
T3PickPlaceCounterToStove
T4PickPlaceSinkToCounter
T5PickPlaceDrawerToCounter
T6PickPlaceCounterToCabinet
T7PickPlaceToasterToCounter
T8TurnOnSinkFaucet
T9CoffeeSetupMug
Figure 8: Cross-task scaling from the Goal-S5 source task. Cumulative Critic–Recovery capabilities discovered on Goal-S5 are transferred to Goal-S3, Goal-S4, and Goal-S9. Within each task, all four arms use the same fixed seeds, policy RNGs, checkpoint, and execution budget. Shaded regions denote Wilson 95% confidence intervals.
Figure 8: Cross-task scaling from the Goal-S5 source task. Cumulative Critic–Recovery capabilities discovered on Goal-S5 are transferred to Goal-S3, Goal-S4, and Goal-S9. Within each task, all four arms use the same fixed seeds, policy RNGs, checkpoint, and execution budget. Shaded regions denote Wilson 95% confidence intervals.
Figure 9: Cumulative reflection scaling across 18 RoboCasa tasks. The macro-average success rate increases from 73.56% for the frozen parent harness to 78.71%, 84.85%, 90.54%, and 93.56% after four cumulative global repair rounds. Each checkpoint retains the previously validated critic, recovery, and tool capabilities.
Figure 9: Cumulative reflection scaling across 18 RoboCasa tasks. The macro-average success rate increases from 73.56% for the frozen parent harness to 78.71%, 84.85%, 90.54%, and 93.56% after four cumulative global repair rounds. Each checkpoint retains the previously validated critic, recovery, and tool capabilities.
Table 5: Mapping of LIBERO-Pro task identifiers used in the main text to the underlying LIBERO tasks. “Goal” corresponds to LIBERO-Goal and “Long” corresponds to LIBERO-10. The same task indices are used for both T and S perturbation settings.
IDLIBERO-GoalIDLIBERO-10 (Long)
Task 0open the middle drawer of the cabinetTask 0put both the alphabet soup and the tomato sauce in the basket
Task 1put the bowl on the stoveTask 1put both the cream cheese box and the butter in the basket
Task 2put the wine bottle on top of the cabinetTask 2turn on the stove and put the moka pot on it
Task 3open the top drawer and put the bowl insideTask 3put the black bowl in the bottom drawer of the cabinet and close it
Task 4put the bowl on top of the cabinetTask 4put the white mug on the left plate and put the yellow and white mug on the right plate
Task 5push the plate to the front of the stoveTask 5pick up the book and place it in the back compartment of the caddy
Task 6put the cream cheese in the bowlTask 6put the white mug on the plate and put the chocolate pudding to the right of the plate
Task 7turn on the stoveTask 7put both the alphabet soup and the cream cheese box in the basket
Task 8put the bowl on the plateTask 8put both moka pots on the stove
Task 9put the wine bottle on the rackTask 9put the yellow and white mug in the microwave and close it
Figure 10: Reflection-driven scaling and transfer on PnP tasks. On PnP-Stove, successive rounds add object-relative pregrasp alignment, bounded regrasp after grasp loss, and stable placement. The lower panels apply the cumulative checkpoints to PnP-Sink, PnP-Cabinet, and PnP-Toaster without an additional training or evolution loop. The right-hand panels show the failure signatures identified by the critic and the corresponding recoveries.
Figure 10: Reflection-driven scaling and transfer on PnP tasks. On PnP-Stove, successive rounds add object-relative pregrasp alignment, bounded regrasp after grasp loss, and stable placement. The lower panels apply the cumulative checkpoints to PnP-Sink, PnP-Cabinet, and PnP-Toaster without an additional training or evolution loop. The right-hand panels show the failure signatures identified by the critic and the corresponding recoveries.
Figure 11: Reflection-driven scaling and transfer on articulated interaction tasks. On TurnOffStove, successive rounds add target localization, collision-aware pre-contact approach, and stable EEF–target contact. The lower panels apply the cumulative checkpoints to TurnOnSinkFaucet, OpenCabinet, and TurnOnMicrowave without an additional training or evolution loop. The right-hand panels show the diagnosed target, approach, and contact failures together with the recovered states.
Figure 11: Reflection-driven scaling and transfer on articulated interaction tasks. On TurnOffStove, successive rounds add target localization, collision-aware pre-contact approach, and stable EEF–target contact. The lower panels apply the cumulative checkpoints to TurnOnSinkFaucet, OpenCabinet, and TurnOnMicrowave without an additional training or evolution loop. The right-hand panels show the diagnosed target, approach, and contact failures together with the recovered states.

为什么重要

如今的机器人AI模型因为训练数据有限,在陌生情况下容易失败,这项工作说明不需要重新训练模型,只靠进化执行过程中的监控和纠错规则就能大幅提升成功率。这为降低机器人训练成本提供了实用路径,而且在一个任务上学到的纠错技能可以直接迁移到相似任务上,无需额外训练。

本文术语

  • VLA(视觉-语言-动作模型) · 接收摄像头画面和语言指令、直接输出机器人动作的人工智能模型
  • rollout(一次执行/推演) · 机器人从开始到结束完整尝试一次任务的过程
  • 闭环与开环 · 闭环指执行过程中持续检查并做出反应,开环指按固定计划执行、结束后才检查
  • critic(评判器) · 实时监视机器人执行状态、发现问题迹象的代码模块
  • recovery skill(恢复技能) · 检测到问题后用于撤销或重试某一步的应对动作

论文原文摘要（英文）

Embodied agents are increasingly used to close the gap left by end-to-end policy models. Yet the agentic path has not realized closed-loop learning in physical execution: existing harnesses remain largely open-loop, following fixed skills during rollout and reflecting only after an episode completes. Such post-hoc reflection cannot govern execution as it unfolds, because physical interaction requires decisions to track rapidly changing robot-environment states at a frequency beyond today's large agentic models. We present Zetta, a closed-loop embodied harness that evolves code-based runtime critics and recovery skills online while keeping the base policy frozen. Through three timescale-separated loops, Zetta provides action-frequency governance, rollout-level critic-recovery proposal, and validation-gated skill updates. Together with Z-Infra, a rollout infrastructure decoupling agent logic from heterogeneous execution resources, Zetta achieves state-of-the-art success on LIBERO-Pro and RoboCasa under our current rollout budget, reaching 90.8% and 93.6%, with an 11.1x inference speedup; success continues to scale with self-exploration experience; learned skills transfer zero-shot, and clear robotic "Aha Moments" emerge. These results show that closed-loop harness self-evolution opens a scaling path for reliable physical intelligence.

作者 · Xin Ding

在 arXiv 阅读

最新论文

全部论文 →

METAL MEDIA 最新报道

图片来源： Xin Ding et al., arXiv:2608.16590, CC BY 4.0