You are a strict filter for navigation instructions.
- Pure action = a movement-only command without a target object/landmark/destination, e.g., ”go straight”, ”turn left”.
- Not pure action = includes a target or landmark, e.g., ”go straight to the bed”.
If there are 4 or more consecutive pure actions anywhere in the instruction, output DROP. Otherwise output KEEP. Consecutive means back-to-back pure actions with no non-pure-action instruction between them.
You are an expert at summarizing navigation trajectories. Given a task and image sequences, your job is to summarize the trajectory from image 0→k in ONE continuous paragraph. Explicitly state which part of the task has been completed. Do NOT mention future parts. Return ONLY a JSON object.
User Prompt Template
Task: {task}
Summary ranges: {segment_list_text}
For EACH range [0 → k], write ONE paragraph in first person. Describe what happened and which part of the task is completed by that point.
Table 9: Stage 2: Current Observation Analysis
System Prompt
You are a controlled reasoning module. Describe the indoor scene layout (open paths, obstacles, landmarks) based on the attached front, left, back, and right views. Only describe visible objects. Do NOT mention targets, goals, history, or next steps.
Output Format
"Analyze the current observations": [your detailed description]
Table 10: Stage 3: Future Action Reasoning (Infer Next Step)
System Prompt
IRON RULE: IF gt_action == "stop", declare arrival. ELSE, reason step-by-step how the move advances the task. The final sentence MUST name the target view and an object-defined region. FORBIDDEN: coordinates, pixels, red circles, proportions.
Target Guidance: {target_guidance} (Use this to locate the region, but do NOT mention the guidance itself).
Table 11: Stage 4: CoT Fusion and Polish
System Prompt
Rewrite the three-stage reasoning into a single, coherent English paragraph. Preserve spatial logic and the target view. Remove redundancies. Do NOT include stage titles. Output ONLY the polished paragraph.
Figure 1: The architecture of TAMP-Nav. Given visual-textual inputs, the agent compresses long-horizon history into an Anchor-Trajectory Memory, autonomously triggering reasoning at critical nodes. As a visual pointer, it selects the optimal multi-camera view and predicts a 2D pixel, which is projected into 3D for SLAM execution. Finally, the policy is optimized via Two-Level GRPO.
Figure 2: The Two-Level GRPO paradigm. TAMP-Nav superimposes trajectory-level rollouts (competing for global navigation success) with step-level candidate rollouts (exploring diverse 2D visual-spatial actions via VLM temperature sampling).
Figure 3: Spatial Heatmap of Reasoning (CoT) Triggers. Comparison between the SFT model (left) and the RL-aligned TAMP-Nav (right).
Figure 4: Performance distribution on long-horizon navigation tasks. The bars show the trajectory count in each length bin, and the curves show the Success Rate (SR) of the evaluated methods as trajectory length increases.
Figure 5: Learning curves of success rewards under different GRPO configurations. The plot illustrates the training progress over 800 steps for three variants: the full Two-Level GRPO, the version without annealed guided sampling (“without guidance”), and the standard GRPO using only trajectory-level rewards (“only trajectory reward”).
Figure 6: Visualization of real-world experiments
Figure 9: Instruction: “Find the red bag”
Figure 15: Visualization of STI token. We employ a clustering algorithm (23) to map high-dimensional embeddings into a 3D space.
Figure 16: Visualization of a simulated navigation trajectory. The agent successfully executes the complex instruction: “Go up the stairs and turn left. Wait at the doorway to the bedroom straight ahead.” by triggering selective reasoning at key decision points and outputting point-and-click actions.
Although Large Vision-Language Models (VLMs) have significantly advanced embodied navigation, their direct deployment remains challenging, as existing methods often force VLMs into unnatural action spaces that misalign with their 2D pre-training priors, compounded by rigid reasoning schedules and inefficient memory management. To overcome these limitations, we propose TAMP-Nav, a unified framework for efficient embodied navigation. First, we introduce a Pixel-to-3D Action Formulation (Point) that reformulates navigation into 2D visual prompting. Specifically, the VLM merely selects 2D pixels, which are then projected into 3D coordinates for a low-level SLAM controller. This design naturally aligns embodied execution with the VLM's inherent 2D visual capabilities. Second, we propose an integrated Selective Reasoning and Anchor-Trajectory Memory mechanism (Think and Memorize), which dynamically triggers Chain-of-Thought and retains high-fidelity memory only at critical nodes, compressing redundant trajectories into lightweight Space-Time Indicators, thereby preserving critical historical information and enhancing spatio-temporal perception. Finally, we design an efficient Two-Level Alignment Paradigm (Align) via Group Relative Policy Optimization (GRPO). By superimposing global outcome rewards with fine-grained process rewards, this dense supervision tightly aligns the agent's cognitive planning with physical environmental feedback, endowing the model with adaptive reasoning capabilities. Experiments demonstrate that TAMP-Nav achieves state-of-the-art performance (e.g., 66.2% SR on R2R-CE) with high runtime and sample efficiency (requiring only 90k training trajectories).
