Spark-to-Paper: End-to-End Research Paper Generation as a Composable Skill

arXiv:2608.119242026-08-11

A system that writes a full research paper end-to-end using only 13 skills inside an existing coding assistant, and revises its own claims when experiments don't support them

Spark-to-Paper turns a research idea into a complete paper using thirteen composable skills inside an existing coding assistant like Claude Code, with no separate agent platform. It commits to an experiment design before seeing results, then revises manuscript claims according to what was actually measured, while splitting error-checking into deterministic gates and model-based self-critique. Across eight controlled research topics it reached 99.5% citation validity and 96.4% figure editability, and fabrication detection rose from 14% for a single-pass draft to 92% with the full integrity and review stack.

METAL MEDIA explanatory visual

Structure of the Spark-to-Paper pipeline

Evidence statusMeasured results reported

Input Routing (Stage 0)Determines whether the input is a short idea or a developed proposal, and whether measured data is available (Proposal vs Data-Aware Mode)
Planning-Citation-Writing-Refinement-Review-Figures-Assembly (Stages 1-7)Thirteen skills read and write shared project artifacts to build the blueprint, bibliography, manuscript, and figures in sequence
Integrity and self-critique layerDeterministic gates (citation, structure, compilation checks) plus Self-Review and Adversarial Review cycle experiment-critique-revision up to seven times
Experiment Execution (Stage 8, conditional)Runs real experiments when code and data are available, records measurements, and updates claims from abstract to conclusion accordingly
Self-Refutation Loop handlingIf seven cycles fail to support the original hypothesis, the trajectory is recorded as a failure report and the pipeline restarts with a new idea

An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.

What they did

The system implements the whole idea-to-paper process (literature retrieval, experiment design and execution, evidence-based claim revision, publication-ready figures, long-horizon consistency) as thirteen composable skills inside an existing coding assistant, without a separate orchestration service.
It separates model judgment (organizing arguments, assessing relevance, evaluating whether evidence supports a claim) from deterministic operations (structure checks, citation validation, LaTeX compilation, plotting measured data) that can be executed and verified mechanically.
Experiment planning is locked in before results are observed, like a lightweight preregistration, and after execution the system reclassifies claims (supported, partially-supported, unsupported, contradicted, needs-confirmation) and propagates revisions across the abstract, introduction, results, and conclusion.
It names a failure mode called the Self-Refutation Loop, where repeated experiment-critique-revision cycles keep rejecting the original research objective, and bounds it at seven cycles before recording a failure report and restarting with a new idea.
Figures for measured results are plotted directly from data into native vector PDFs, while explanatory diagrams are generated with an image model and then reconstructed through code into editable vector formats.

Figure 1: Qualitative positioning against directly related systems, on two axes derived from Table 1. Coordinates are the authors’ own ordinal assessment from each system’s public documentation, not a measured benchmark.

Table 1: Qualitative capability comparison against directly related systems, based on each system’s own public documentation (paper or repository) as of this writing. ✓ full, ⚫ partial, – not offered or not documented.
System	End-to-end	Runs exper.	Draws figures	Editable vectors	No standing infra.
AI Scientist / v2 [13, 28]	✓	✓	✓	–	–
AutoResearchClaw [12]	✓	✓	✓	–	–
Kosmos / Robin [15, 7]	⚫	✓	⚫	–	–
Idea2Story [27]	–	–	⚫	–	✓
ARS [26]	⚫	–	–	–	✓
CycleResearcher [25]	⚫	✓	–	–	–
Spark-to-Paper (ours)	✓	✓	✓	✓	✓

Figure 2: Overview of Spark-to-Paper execution. Stage 0 selects the result-integrity mode, Stages 1–7 coordinate through persistent project artifacts, and conditional Stage 8 writes measured evidence back into the manuscript. Model judgment governs context-dependent decisions, while deterministic tools execute checkable operations.

Table 2: Evaluation dimensions and corresponding measurements used to assess the quality and generation efficiency of Spark-to-Paper.
Metric	Measurement
Quality
Citation validity	Resolved references / total references
Fabrication detection	Detected unsupported claims / injected claims
Figure editability	Editable elements / total figure elements
Review precision	Verified issues / raised review issues
Cross-template robustness	Successful templates / supported templates
Efficiency
Generation cost	Monetary cost, tokens, wall-clock time, and deployment footprint

Figure 3: Integrity and correction in Spark-to-Paper. Deterministic gates enforce verifiable properties, Self-Review and Adversarial Review challenge semantic decisions, and surviving issues trigger revision. Experiment–critique–revision cycles are capped at seven; an unresolved trajectory becomes a failure report rather than a successful manuscript.

Table 3: Main comparison of Spark-to-Paper with human-written preprints, prior autonomous research systems, and a single-pass LLM baseline across artifact quality and generation efficiency metrics.
System	Citation exist. (%)	Fig. editability (%)	Tokens (M)	USD	Wall-clock
Human-written preprints (sampled)	97.8 [94.6, 99.4]	58 [44, 71]	n/a	n/a	n/a
AI Scientist, released papers [13]	93 (42/45)	0 (0/210)	n/r	$10–15 (amort.)	∼12 h / batch
AI Scientist-v2, workshop set [28]	91 (58/64)	3 (0–8)	n/r	∼$20–25 / attempt	≤15 h / run
Agent Laboratory, released paper [20]	96 (27/28)	0 (0/30)	n/r	$2.33 (gpt-4o)	∼19 min
Single-pass LLM draft (same backbone)	81 (range 76–86)	n/a	0.11 (0.09–0.13)	$0.66 (0.55–0.76)	16 min (13–19)
Spark-to-Paper (full stack)	99.5 [98.4, 100]	96.4 [92.7, 98.6]	11.9 [10.2, 13.7]	$8.1 [6.9, 9.6]	3.2 h [2.6, 3.9]

Figure 4: Role-aware editable figure generation. Measured results follow deterministic plotting to native vector PDF, whereas explanatory content uses a raster visual target followed by iterative HTML reconstruction and vector export; unreliable reconstruction falls back to the raster.

Table 4: Ablation study of the Spark-to-Paper quality stack, showing the contribution and incremental cost of gating, self-review, and adversarial review.
Configuration	Fabr. detection (%)	Review precision (%)	Δ tokens (M)	Δ USD
Single-pass draft (no gates)	14 (5/36) [6, 29]	n/a	ref.	ref.
Gates only	69 (25/36) [53, 82]	n/a	+8.1 ± 0.9	+5.3 ± 0.5
Gates + self-review	81 (29/36) [65, 90]	n/a	+1.1 ± 0.2	+0.6 ± 0.1
Gates + self-review + adversarial review	92 (33/36) [78, 97]	74 (42/57) [61, 83]	+2.6 ± 0.4	+1.6 ± 0.2

Figure 5: Page, reference, and figure counts for the end-to-end papers documented in the project’s own showcase materials. Self-reported by the system’s maintainers, not an independent measurement; included as existence evidence, not as a benchmark result.

Table 5: Claim admission labels and corresponding revision actions.
Label	Action
supported	Retain with evidence-matched wording
partially-supported	Narrow the claim or request additional evidence
unsupported	Run a feasible missing experiment, weaken, or remove
contradicted	Remove or report as a limitation
needs-confirmation	Return the unresolved claim for author confirmation

Figure 6: Case study on two different domain demo papers with only one short proposal as inputs, incorrect expectations are highlighted by Bold and Underline text.

Table 6: Main persistent artifacts used by Spark-to-Paper.
Artifact	Stage	Purpose
blueprint.json	Planning	Paper structure, claims, notation, experiments
template.json	Planning	Venue specification and execution mode
refs.bib	Citation	Verified bibliography
claims_map.json	Citation	Claim–citation associations
sections/*.tex	Writing	Section-level manuscript sources
figures/	Figure	Figures and generation records
results.facts.json	Data	Grounded quantitative evidence
main.tex/pdf	Assembly	Final manuscript project
logs/*.io.md	All stages	Stage-level input and output records

Figure 7: Paper Demo (Part 1): Screenshot for the demo paper introduction.

Findings

Across eight controlled research topics, the full system achieved 99.5% citation validity and 96.4% figure editability.
In a controlled ablation with injected unsupported claims, fabrication detection rose from 14% for a single-pass draft to 92% with the complete integrity and review stack.
Adversarial review achieved 74% precision on the issues it raised.
The full system used 11.9M tokens and cost $8.1 per manuscript on average, requiring 3.2 hours of wall-clock time.

Figure 8: Paper Demo (Part 2): Screenshot for the demo paper method.

Where it can be used

Expanding a rough research idea into a complete draft manuscript quickly.
Automatically updating a paper's text, tables, and figures once code and experimental results already exist.
Using the fabrication-detection and citation-validation checks as an auditing aid for draft manuscripts.
Referencing the claim-revision workflow (narrowing or moving claims to limitations) when experimental outcomes diverge from expectations.

Figure 9: Paper Demo (Part 3): Screenshot for the demo paper experiment and analysis.

Limits and open work

Evaluation covers only eight controlled research topics, so generalization to broader fields or real submission conditions is not established.
The comparison against other systems in Figure 1 and Table 1 is the authors' own ordinal assessment from public documentation, not a measured benchmark.
The page, reference, and figure counts for showcase papers in Figure 5 are self-reported by the system's maintainers, not independently measured.
Judging argument persuasiveness, contribution importance, or scientific appropriateness of experimental design still relies on model judgment rather than deterministic verification.
Semantic assessment of whether evidence supports a claim is currently performed by the model, and the accuracy of this specific judgment step is not separately measured.

Figure 10: Paper Demo (Part 4): Screenshot for the demo paper experiment and analysis.

Why it matters

It shows a path for merging the currently separate workflows of running experiments in code and drafting papers in another tool, by keeping everything inside the coding assistant researchers already use. It also gives a concrete, measured design for catching unsupported or fabricated claims, which matters for anyone evaluating the trustworthiness of automated research-writing tools.

Terms in this paper

Deterministic gate · A mechanical check with explicit pass/fail rules, such as citation validation or LaTeX compilation success
Self-Refutation Loop · A failure pattern where repeated experiments keep concluding the original research objective isn't supported, yet the system keeps revising the same direction instead of stopping
Adversarial Review · A manuscript-level stage where multiple independent review passes challenge the paper from different angles, like theoretical soundness or experimental design
Data-Aware Mode / Proposal Mode · Two execution modes selected depending on whether measured experimental data is already available, controlling how strictly quantitative statements must be grounded

Original abstract (English)

Turning a research idea into a complete paper requires more than text generation: the system must retrieve literature, design and execute experiments, revise claims according to evidence, produce publication-ready figures, and maintain consistency across a long generation process. We present Spark-to-Paper, an end-to-end research paper generation system implemented as thirteen composable skills inside an existing coding assistant, without requiring a separate agent platform or orchestration service. Spark-to-Paper separates model-based judgment from deterministic operations that can be directly executed and checked. It further separates experiment planning from reporting, so that required evidence is specified before results are observed and manuscript claims are revised according to measured outcomes. To improve reliability over long research trajectories, the system combines deterministic integrity checks with self-critique and bounds a failure mode we call the Self-Refutation Loop, in which repeated experiments continue to reject the original research objective. Spark-to-Paper also produces editable vector figures through programmatic plotting for experimental results and code-based reconstruction for generated method diagrams. Across eight controlled research topics, Spark-to-Paper achieves 99.5% citation validity and 96.4% figure editability. A controlled ablation increases fabrication detection from 14% for a single-pass draft to 92% with the full integrity and review stack, while adversarial review achieves 74% precision. The full system uses 11.9M tokens, costs $8.1 per manuscript, and requires 3.2 hours on average. These results show that end-to-end research paper generation can be implemented as a lightweight, composable workflow inside existing coding assistants while keeping experimental evidence central to how claims are accepted, revised, or abandoned.

Authors · Zhuoyang Qian

Read on arXiv

Latest papers

All papers →

Latest from METAL MEDIA

NCT 127 drop BLINGY, their first full album in two years

Figures: Zhuoyang Qian et al., arXiv:2608.11924, arxiv-nonexclusive