Everything K-culture — comebacks to K-beauty, straight to your inboxGet it in your inbox

METAL MEDIA

WorldClaw: Agentic 3D Open-World Generation at Scale

arXiv:2608.052482026-08-07

AI agents work as a team to build whole walkable 3D open worlds from a single line of text

WorldClaw is an agentic system that turns one text prompt into a large-scale 3D world complete with terrain, buildings, trees, and ships. Planning agents first turn the prompt into a structured plan of regions, terrain, and objects, then a global terrain is built, and finally only the regions that need detail are populated with generated and placed objects. Render-based agents inspect the results and iteratively fix problems to make the scene coherent.

METAL MEDIA explanatory visual

WorldClaw's three-stage global-to-regional generation flow

Evidence statusMeasured results and planned work

  1. 1. Intent Analysis & PlanningTurns the short user prompt into a structured spec of regions, terrain, objects, materials, and spatial relations
  2. 2. Global Terrain GenerationBuilds a height-field terrain from the semantic layout map and terrain parameters, scattering reusable assets and materials
  3. 3. Regional Object Generation & PlacementRenders detail-needed regions, inserts objects via image editing, reconstructs them as 3D meshes, and places them on the terrain
  4. 4. Render-based Refinement LoopAgents inspect rendered views to fix terrain seams, material scale, and object pose/contact issues iteratively
An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.

What they did

  1. An intent-analysis agent and a scene-planning agent convert a short user prompt into a structured scene specification covering regions, terrain, object categories, materials, and spatial relationships.
  2. From this specification, a color-coded 2D semantic layout map is generated, which drives a height-field terrain, reusable 3D assets (rocks, vegetation), and generative or procedural materials to build a globally coherent terrain foundation.
  3. For regions requiring detail, the system renders the local terrain as a 2D image, uses an image-editing model to insert objects like buildings or ships, then reconstructs each inserted object as an individual 3D mesh and places it back onto the terrain.
  4. Render-based refinement agents inspect rendered views and iteratively fix terrain transitions, material scale mismatches, and object issues such as floating or incorrect scale/pose relative to the terrain.
  5. WorldClaw produced large-scale scenes across varied prompts, including a tropical pirate stronghold, a canyon with tribal settlements, a desert battlefield, and a snow-covered mountain valley, each with coherent regional organization and editable 3D assets.
Figure 1: Overview of WorldClaw. Given an open-ended text prompt, WorldClaw constructs an explicit, explorable, and editable 3D world through a three-stage global-to-regional pipeline. (1) Intent Analysis and Planning translates the prompt into a structured scene specification. (2) Global Terrain Generation establishes a region-aware terrain foundation with coherent geometry, appearance, and spatial semantics. (3) Regional Object Generation and Placement populates planned regions with editable 3D assets and refines their arrangements and interactions with terrain.
Figure 1: Overview of WorldClaw. Given an open-ended text prompt, WorldClaw constructs an explicit, explorable, and editable 3D world through a three-stage global-to-regional pipeline. (1) Intent Analysis and Planning translates the prompt into a structured scene specification. (2) Global Terrain Generation establishes a region-aware terrain foundation with coherent geometry, appearance, and spatial semantics. (3) Regional Object Generation and Placement populates planned regions with editable 3D assets and refines their arrangements and interactions with terrain.
Figure 2: Overview of global terrain generation and refinement.(a) Initial Height-Field Generation constructs composite terrain geometry from the semantic layout map and region-specific terrain parameters and assigns materials to the corresponding regions.(b) Global Terrain Asset Scattering instantiates terrain-associated assets according to regional semantics and local surface conditions.(c) Terrain Refinement renders, inspects, and locally edits the terrain to correct geometric transitions, material scales, asset distributions, and rendering artifacts.
Figure 2: Overview of global terrain generation and refinement.(a) Initial Height-Field Generation constructs composite terrain geometry from the semantic layout map and region-specific terrain parameters and assigns materials to the corresponding regions.(b) Global Terrain Asset Scattering instantiates terrain-associated assets according to regional semantics and local surface conditions.(c) Terrain Refinement renders, inspects, and locally edits the terrain to correct geometric transitions, material scales, asset distributions, and rendering artifacts.

Findings

  • WorldClaw generated large-scale scenes with coherent global spatial organization and rich regional content across diverse prompts, including a tropical pirate stronghold, canyon with tribal settlements, desert battlefield, snow-covered mountain valley, medieval village, snow riverside village, desert adventure camp, and island with Japanese-style towns.
  • A qualitative comparison against representative text-driven 3D scene generation methods was conducted using a shared medieval-village theme, though no numeric metrics were reported for this comparison.
  • The authors observed that current open-source language models often failed to generate procedural terrain and material code that was both executable and consistent with requirements, and open-source image generation models often failed to produce usable layout maps or preserve object appearance and pose.
  • The authors state that fully validating the decoupled pipeline currently still requires capable models such as Claude Opus 4.8, GPT-Image-2, and Hunyuan3D.
Figure 3: Scene refinement. (a) Object Refinement processes objects from a report queue, evaluates pose, mesh quality, and scale against semantic and regional context, applies targeted edits, and verifies the result through re-rendering. (b) Terrain Refinement examines support-surface quality and object-terrain collisions, applies local co-deformation to defects such as floating, and updates the report after re-rendering.
Figure 3: Scene refinement. (a) Object Refinement processes objects from a report queue, evaluates pose, mesh quality, and scale against semantic and regional context, applies targeted edits, and verifies the result through re-rendering. (b) Terrain Refinement examines support-surface quality and object-terrain collisions, applies local co-deformation to defects such as floating, and updates the report after re-rendering.
Figure 4: Tropical pirate stronghold. An island terrain organizes dense vegetation, settlements, docks, and ships into distinct coastal regions. The figure presents the global composition, regional close-ups, local walk views, and their corresponding instance, depth, and normal renderings.
Figure 4: Tropical pirate stronghold. An island terrain organizes dense vegetation, settlements, docks, and ships into distinct coastal regions. The figure presents the global composition, regional close-ups, local walk views, and their corresponding instance, depth, and normal renderings.

Where it can be used

  • Quickly drafting large-scale terrain and building layouts for game or VR environments from text alone
  • Preparing large outdoor previsualization backgrounds for film or animation as editable 3D assets
  • Generating diverse 3D environments with varied terrain and object placements for robot simulation or embodied-AI training
  • Producing textured-mesh source content that can be imported into game engines like Unreal Engine for further editing and animation
Figure 5: Canyon with tribal settlements. A continuous river connects the canyon, valley floor, vegetation, and settlement regions across substantial elevation changes. Global and regional views are complemented by local walk views and the corresponding instance, depth, and normal renderings.
Figure 5: Canyon with tribal settlements. A continuous river connects the canyon, valley floor, vegetation, and settlement regions across substantial elevation changes. Global and regional views are complemented by local walk views and the corresponding instance, depth, and normal renderings.
Figure 6: Thrilling battlefield in the desert. Layered rocky landforms surround open combat areas and populated compounds containing buildings, defensive structures, and vehicles. The figure shows the global and regional organization together with local walk views and their instance, depth, and normal renderings.
Figure 6: Thrilling battlefield in the desert. Layered rocky landforms surround open combat areas and populated compounds containing buildings, defensive structures, and vehicles. The figure shows the global and regional organization together with local walk views and their instance, depth, and normal renderings.

Limits and open work

  • Each pipeline stage depends heavily on external LLM, image-generation, and 3D-generation models, and the final visual quality is directly bounded by the quality of these backbone models.
  • LLM-generated Blender code frequently contains errors in scale estimation, numerical parameters, or node connectivity, causing inconsistent landforms, inaccurate materials, or misplaced objects that require multiple render-inspect-refine iterations to fix.
  • The pipeline's separate generation and repeated refinement loops for terrain, assets, and contacts incur substantial inference latency and computational cost, which grows with the number of objects and refinement iterations.
  • For simpler scenes, this long-horizon pipeline can be unnecessarily inefficient compared to holistic generation methods that need fewer steps.
  • Because objects are still reconstructed mainly via generative 3D models, the system does not consistently recover explicit part hierarchies, parametric structure, articulation, or interaction logic, which the authors flag as future work.
Figure 7: Snow-covered mountain valley with style of Command & Conquer: Red Alert. The enclosing mountain terrain contains multiple regions populated with futuristic facilities, communication structures, and vehicles. Global, regional, and local walk views are shown with the corresponding instance, depth, and normal renderings.
Figure 7: Snow-covered mountain valley with style of Command & Conquer: Red Alert. The enclosing mountain terrain contains multiple regions populated with futuristic facilities, communication structures, and vehicles. Global, regional, and local walk views are shown with the corresponding instance, depth, and normal renderings.
Figure 8: Qualitative comparison with the representative text-driven 3D scene generation methods. All methods are conditioned on prompts that share the same medieval-village theme and comparable scene requirements, with the wording adapted when necessary to each method’s input format.
Figure 8: Qualitative comparison with the representative text-driven 3D scene generation methods. All methods are conditioned on prompts that share the same medieval-village theme and comparable scene requirements, with the wording adapted when necessary to each method’s input format.

Why it matters

Games, film, VR, and robot simulation all need explicit, walkable, editable 3D worlds, but existing methods struggle to keep the whole scene coherent while also filling it with rich local detail. WorldClaw's global-then-local, agentic approach is a concrete attempt to bridge that gap, offering a reference workflow for turning open-ended text directly into production-usable 3D content.

Figure 9: Medieval-style village across diverse terrain. The scene combines snow-capped mountains, forested plains, waterways, and desert areas, with village buildings, windmills, vegetation, and animals distributed across the different regions. The figure shows the global layout, regional close-ups, local walk views, and their corresponding instance, depth, and normal renderings.
Figure 9: Medieval-style village across diverse terrain. The scene combines snow-capped mountains, forested plains, waterways, and desert areas, with village buildings, windmills, vegetation, and animals distributed across the different regions. The figure shows the global layout, regional close-ups, local walk views, and their corresponding instance, depth, and normal renderings.
Figure 10: Snow-covered riverside village. The village extends along both sides of a frozen river within a mountainous landscape. The figure shows the global layout, regional close-ups, and local walk views together with instance, depth, and normal renderings.
Figure 10: Snow-covered riverside village. The village extends along both sides of a frozen river within a mountainous landscape. The figure shows the global layout, regional close-ups, and local walk views together with instance, depth, and normal renderings.

Terms in this paper

  • Agentic · AI systems that plan, execute tools, inspect results, and revise on their own rather than producing output in a single pass
  • Height field · A representation of terrain elevation as numeric values over a 2D grid
  • Semantic layout map · A 2D map using distinct colors to mark what terrain type or function each region has
  • Image-to-3D · A model capability that takes a 2D image of an object and reconstructs its 3D shape and appearance
  • Textured mesh · An editable 3D shape with surface color and texture applied, usable in standard 3D tools
Figure 11: Desert adventure camp surrounded by dragons. The scene combines layered desert terrain, settlements, watchtowers, and dragons distributed around the camp. Global, regional, and local walk views are shown with the corresponding instance, depth, and normal renderings.
Figure 11: Desert adventure camp surrounded by dragons. The scene combines layered desert terrain, settlements, watchtowers, and dragons distributed around the camp. Global, regional, and local walk views are shown with the corresponding instance, depth, and normal renderings.
Figure 12: Island with Japanese-style towns. Multiple settlements are distributed across an island containing coastlines, vegetation, hills, and water. The figure presents the global and regional organization as well as local walk views and their instance, depth, and normal renderings.
Figure 12: Island with Japanese-style towns. Multiple settlements are distributed across an island containing coastlines, vegetation, hills, and water. The figure presents the global and regional organization as well as local walk views and their instance, depth, and normal renderings.

Original abstract (English)

Generating large-scale, freely explorable 3D worlds from open-ended text remains challenging because a system must jointly maintain global spatial coherence, rich local content, and explicit assets suitable for downstream editing and reuse. We present WorldClaw, a fully agentic, coarse-to-fine framework for open-world 3D scene generation. Planning agents translate a text prompt into a structured specification of regions, terrain, assets, materials, and spatial relations. WorldClaw then builds a globally coherent terrain foundation from semantic layouts, reusable assets, generative or procedural materials, and a region-aware height field. For detail-demanding regions, it generates terrain-conditioned compositions, reconstructs editable textured meshes, and recovers their placement on the terrain; render-based agents further refine terrain, objects, appearance, and contacts. Across diverse open-world prompts, WorldClaw produces large-scale scenes with coherent spatial organization, visually compelling local content, and editable instance-level assets while preserving a consistent global terrain structure.

Authors · Chunchao Guo, Jinpeng Li, Yang Li, Zilong Huang

Read on arXiv

Latest papers

All papers →

Latest from METAL MEDIA

Figures: Chunchao Guo et al., arXiv:2608.05248, arxiv-nonexclusive