Figure 1: Overview of FACET. Stage 1 collects skills and constructs the scenario–skill repository. Stage 2 understands the selected skills, explores and recovers a joint scenario, expands it into a complete representation, and builds aligned solution and instruction references. Stage 3 constructs and validates the environment before generating the final task artifacts, with bounded repair loops for failed builds and tests.
Table 1: Trajectory- and task-level comparison of terminal-agent datasets. Trajectories are collected using the Terminus-2 scaffold, and task performance is evaluated using DeepSeek-V4-Pro with Terminus-2. Turns and Tests denote the average interaction turns per trajectory and executable checkpoints per task. Detailed protocols are provided in Appendix A.6.
Dataset
Trajectory
Task
#Traj.
Turns
#Tasks
Tests
P@1
P@3
Nemotron-Terminal (29)
5K
6.12
15K
6.18
40.67
48.00
Endless-Terminals (13)
200
4.53
2,492
5.51
83.00
87.00
Terminal-Lego (33)
32K
5.77
15K
16.60
47.00
49.00
TerminalWorld (8)
200
11.94
1,530
3.98
57.00
82.00
Tmax (16)
500
11.14
15K
3.29
80.00
86.00
FACET (ours)
1.2K
11.86
6K
22.77
27.00
35.00
Figure 2: Analysis of execution and synthesis failures. (a) Distribution of failed or errored verifier checks among unsuccessful teacher rollouts; three rollouts without a parsed FAILED/ERROR result are omitted. (b) Distribution of initial validation failure types under the Forward, Reverse, and Joint generation schemes.
Table 2: Results on Terminal-Bench 2.1. Scores under our evaluation setting are averaged over three independent attempts per task.
Model
Size
Agent
Terminal-Bench 2.1
Reported reference models
GPT-5.5 (xhigh) (31)
—
Terminus-2
78.00
Claude Opus 4.7 (max) (31)
—
Terminus-2
66.10
Gemini 3 Pro (high) (31)
—
Gemini CLI
65.80
Intern-S2-Preview-397B (7)
397B
Terminus-2
67.42
MiniMax M3 (19)
428B
Terminus-2
66.00
GLM-5.1 (max) (31)
744B
Claude Code
58.70
Models evaluated under our setting
Qwen3.6-27B
27B
Terminus-2
53.93
Qwen3.5-397B-A17B
397B
Terminus-2
49.06
Kimi-K2.6
1T
Terminus-2
59.93
DeepSeek-V4-Pro-Preview (high)
1.6T
Terminus-2
73.03
Qwen3.5 base models
Qwen3.5-4B
4B
Terminus-2
17.60
Qwen3.5-9B
9B
Terminus-2
27.34
Qwen3.5-27B
27B
Terminus-2
40.82
Fine-tuned models
FACET-Terminal-Qwen3.5-4B
4B
Terminus-2
24.72 (+7.12)
FACET-Terminal-Qwen3.5-9B
9B
Terminus-2
35.58 (+8.24)
FACET-Terminal-Qwen3.5-27B
27B
Terminus-2
47.57 (+6.75)
Figure 3: Distributions of source skills and synthesized tasks. (a) The retained skill corpus spans five top-level families and 34 fine-grained categories. (b) The 6,078 validated tasks are distributed across nine task families, whose individual shares range from 9.59% to 11.99%.
Table 3: Top-level source-skill distribution.
Category
Skills
Share
AI, agents, and tools
15,182
21.28%
Software, systems, and security
15,059
21.11%
Data, analysis, and research
12,409
17.39%
Documents, productivity, and workflows
11,267
15.79%
Multimedia, creation, and publishing
17,424
24.42%
Total
71,341
100.00%
Figure 4: Strict task-level pass rates for the 20 most frequent skill tags among the 6,066 completed rollouts. Error bars indicate Wilson 95% confidence intervals, and the dashed line denotes the overall pass rate. Tags are selected by frequency and ordered by observed pass rate.
Table 4: Fine-grained source-skill categories. “Global” is the share of all 71,341 skills; “within parent” is the share inside the corresponding top-level category.
Parent
Fine-grained category
Count
Global
Within parent
AI, agents, and tools
Agent orchestration and automation
2,782
3.90%
18.32%
Prompt and model calls
3,310
4.64%
21.80%
Skills, plugins, and extensions
1,637
2.29%
10.78%
MCP and external tools
2,579
3.62%
16.99%
Memory, RAG, and knowledge bases
3,439
4.82%
22.65%
Multi-agent collaboration
1,435
2.01%
9.45%
Software, systems, and security
Code generation and development
3,065
4.30%
20.35%
Testing, debugging, and code quality
2,167
3.04%
14.39%
Git, build, and dependency management
1,904
2.67%
12.64%
Deployment, containers, and DevOps
2,069
2.90%
13.74%
System administration and CLI
3,576
5.01%
23.75%
Security, privacy, and compliance
2,278
3.19%
15.13%
Data, analysis, and research
JSON, YAML, and XML
2,752
3.86%
22.18%
CSV, Excel, and spreadsheets
1,544
2.16%
12.44%
Databases and SQL
1,627
2.28%
13.11%
Data cleaning, conversion, and validation
1,595
2.24%
12.85%
Statistical analysis and metrics
1,892
2.65%
15.25%
Visualization and dashboards
1,419
1.99%
11.44%
Search, research, and extraction
1,580
2.21%
12.73%
Documents, productivity, and workflows
Documents and Markdown
1,496
2.10%
13.28%
Reports, summaries, and briefs
1,865
2.61%
16.55%
PDF, Office, and presentations
1,429
2.00%
12.68%
Office and personal productivity
1,592
2.23%
14.13%
Project, task, and schedule management
1,682
2.36%
14.93%
System integration and automation
1,597
2.24%
14.17%
Audit, checklists, and operation records
1,606
2.25%
14.25%
Multimedia, creation, and publishing
Image generation and editing
2,699
3.78%
15.49%
Design, drawing, and visual assets
1,858
2.60%
10.66%
Audio, speech, and music
1,917
2.69%
11.00%
Video, animation, and captions
2,491
3.49%
14.30%
Figure 5: Command-level patterns in successful teacher trajectories. The left panel shows the most frequent shell commands, colored by their dominant contextual class. The right panel reports row-normalized transitions between adjacent assistant-turn states.
Table 5: Task-construction funnel. Percentages in the last column use the immediately preceding comparable stage.
Stage
Count
Stage retention
Scenario–skill seeds
7,852
—
Seeds with first-build logs
7,841
99.86%
Initial environment success
6,630
84.56%
Environment repair recovery
874
—
Successful environments
7,504
95.70%
Entering task validation
7,446
99.23%
First-pass valid tasks
2,856
38.35%
Task repair recovery
3,222
—
Final validated tasks
6,078
81.63%
Table 6: Selected command- and turn-level statistics for successful teacher trajectories. Shares for command statistics use all 39,136 command occurrences; transition probabilities are normalized within the current turn state.
View
Statistic
Value
Command usage
cat occurrences
18,168 (46.4%)
Top three commands
27,207 (69.5%)
Top ten commands
34,598 (88.4%)
Observation commands
33,140 (84.7%)
Turn dynamics
Observation-only first turn
1,214 (95.6%)
Observation-only → observation-only
4,324 (55.6%)
Action-only → observation-only
1,399 (53.1%)
Action-only → action-only
758 (28.7%)
Table 7: Outcomes of three artifact-generation orders on 100 shared semantic paths. Initial validity is computed over tasks that reach validation, while final yield is computed over all selected paths.
Scheme
Reached validation
Initially valid
Final yield
Forward (Ours)
99
46 (46.5%)
83/100
Reverse
91
22 (24.2%)
63/100
Joint
96
36 (37.5%)
65/100
Table 8: Shared-state task construction and Docker round-trip validation.
Step
Stage
Operation
1
Materialize environment
Generate the Dockerfile, fixtures, dependencies, and initialization scripts from the reconstructed task specification.
2
Build and repair
Build the image with docker build. Build or initialization failures are returned to the environment-repair agent for at most three iterations.
3
Capture shared state
Start a temporary container, inspect the task workspace, and record the realized initial state e0.
4
Generate artifacts
Generate the final instruction, reference solution, and verifier using the same reconstructed specification and shared state e0.
5
Baseline validation
Start a clean container, copy and execute the verifier without running the solution, and require reward 0.
6
Oracle validation
Start another clean container, copy and execute solution/solve.sh, run tests/test.sh, and require reward 1.
7
Repair and revalidate
Classify a failure as an instruction, environment, solution, or verifier defect, apply the corresponding repair, and repeat the full validation procedure for at most five rounds.
8
Accept and clean up
Retain the task only after all validation conditions pass, then stop and remove temporary containers and images.
Table 9: Principal training and evaluation configurations.
Setting
Value
Supervised fine-tuning
Base models
Qwen3.5-4B, Qwen3.5-9B, Qwen3.5-27B
Training data
1.2K complete successful trajectories
Training strategy
Full-parameter SFT with BF16 and ZeRO-3
Epochs / effective batch size
3 / 64
Learning rate / schedule
1×10−5 / cosine with 0.1 warmup ratio
Maximum sequence length
32,768 tokens
Evaluation
Benchmark / agent
Terminal-Bench 2.1 / Terminus-2
Attempts / timeout
3 per task / 2 hours per attempt
Temperature
1.0
Context length
FACET models: 32,768 tokens; other models: officially supported maximum
Table 10: End-to-end comparison over 500 common skill-pair inputs. Packages denotes complete Harbor task packages, Validated denotes tasks passing oracle validation, and Yield is computed over all inputs. P@1 and P@3 are evaluated on the tasks retained by each pipeline, and Avg. Cmds. is the average number of terminal commands per rollout.
Training terminal agents requires scalable executable supervision, yet synthesizing high-quality terminal tasks remains challenging. Each task couples an instruction, an initialized environment, a reference solution, and an executable verifier; if these artifacts are generated from inconsistent assumptions, the resulting task may be unsolvable or incorrectly evaluated. Meanwhile, multi-stage synthesis can discard the goals, dependencies, state transitions, and procedural constraints encoded in the original sources. We present FACET (Fine-grained Agentic Construction of Executable Tasks), a framework that addresses both information preservation and cross-artifact consistency. FACET reconstructs related agent skills into coherent, information-rich scenarios, then realizes and repairs the execution environment before generating the final task artifacts. The resulting container state serves as shared grounding for the instruction, solution, and verifier, while execution-based validation and targeted repair correct artifact-specific failures without unnecessarily regenerating valid components. FACET produces complex terminal tasks with dense executable checks, and successful trajectories collected from these tasks provide effective, data-efficient supervision. Fine-tuning models across multiple scales consistently improves performance on Terminal-Bench 2.1, while analyses of alternative generation schemes support the importance of environment-grounded construction for task validity and solution-verifier alignment. These results establish source-intent preservation and shared executable-state grounding as key principles for scalable terminal-task synthesis.
作者 · Kou Shi, Zun Wang, Qisheng Su, Shiting Huang, Ziao Zhang, Zhen Fang, Qingnan Ren, Jin Liu, Yu Zeng, Yiming Zhao, Lin Che
