Everything K-culture — comebacks to K-beauty, straight to your inboxGet it in your inbox

METAL MEDIA

Spark-to-Paper: End-to-End Research Paper Generation as a Composable Skill

arXiv:2608.119242026-08-11

A system that writes a full research paper end-to-end using only 13 skills inside an existing coding assistant, and revises its own claims when experiments don't support them

Spark-to-Paper turns a research idea into a complete paper using thirteen composable skills inside an existing coding assistant like Claude Code, with no separate agent platform. It commits to an experiment design before seeing results, then revises manuscript claims according to what was actually measured, while splitting error-checking into deterministic gates and model-based self-critique. Across eight controlled research topics it reached 99.5% citation validity and 96.4% figure editability, and fabrication detection rose from 14% for a single-pass draft to 92% with the full integrity and review stack.

METAL MEDIA explanatory visual

Structure of the Spark-to-Paper pipeline

Evidence statusMeasured results reported

  1. Input Routing (Stage 0)Determines whether the input is a short idea or a developed proposal, and whether measured data is available (Proposal vs Data-Aware Mode)
  2. Planning-Citation-Writing-Refinement-Review-Figures-Assembly (Stages 1-7)Thirteen skills read and write shared project artifacts to build the blueprint, bibliography, manuscript, and figures in sequence
  3. Integrity and self-critique layerDeterministic gates (citation, structure, compilation checks) plus Self-Review and Adversarial Review cycle experiment-critique-revision up to seven times
  4. Experiment Execution (Stage 8, conditional)Runs real experiments when code and data are available, records measurements, and updates claims from abstract to conclusion accordingly
  5. Self-Refutation Loop handlingIf seven cycles fail to support the original hypothesis, the trajectory is recorded as a failure report and the pipeline restarts with a new idea
An explanatory diagram made by METAL MEDIA, not a figure supplied by the paper's authors.

What they did

  1. The system implements the whole idea-to-paper process (literature retrieval, experiment design and execution, evidence-based claim revision, publication-ready figures, long-horizon consistency) as thirteen composable skills inside an existing coding assistant, without a separate orchestration service.
  2. It separates model judgment (organizing arguments, assessing relevance, evaluating whether evidence supports a claim) from deterministic operations (structure checks, citation validation, LaTeX compilation, plotting measured data) that can be executed and verified mechanically.
  3. Experiment planning is locked in before results are observed, like a lightweight preregistration, and after execution the system reclassifies claims (supported, partially-supported, unsupported, contradicted, needs-confirmation) and propagates revisions across the abstract, introduction, results, and conclusion.
  4. It names a failure mode called the Self-Refutation Loop, where repeated experiment-critique-revision cycles keep rejecting the original research objective, and bounds it at seven cycles before recording a failure report and restarting with a new idea.
  5. Figures for measured results are plotted directly from data into native vector PDFs, while explanatory diagrams are generated with an image model and then reconstructed through code into editable vector formats.
Figure 1: Qualitative positioning against directly related systems, on two axes derived from Table 1. Coordinates are the authors’ own ordinal assessment from each system’s public documentation, not a measured benchmark.
Figure 1: Qualitative positioning against directly related systems, on two axes derived from Table 1. Coordinates are the authors’ own ordinal assessment from each system’s public documentation, not a measured benchmark.
Table 1: Qualitative capability comparison against directly related systems, based on each system’s own public documentation (paper or repository) as of this writing. ✓ full, ⚫ partial, – not offered or not documented.
SystemEnd-to-endRuns exper.Draws figuresEditable vectorsNo standing infra.
AI Scientist / v2 [13, 28]✓✓✓––
AutoResearchClaw [12]✓✓✓––
Kosmos / Robin [15, 7]⚫✓⚫––
Idea2Story [27]––⚫–✓
ARS [26]⚫–––✓
CycleResearcher [25]⚫✓–––
Spark-to-Paper (ours)✓✓✓✓✓
Figure 2: Overview of Spark-to-Paper execution. Stage 0 selects the result-integrity mode, Stages 1–7 coordinate through persistent project artifacts, and conditional Stage 8 writes measured evidence back into the manuscript. Model judgment governs context-dependent decisions, while deterministic tools execute checkable operations.
Figure 2: Overview of Spark-to-Paper execution. Stage 0 selects the result-integrity mode, Stages 1–7 coordinate through persistent project artifacts, and conditional Stage 8 writes measured evidence back into the manuscript. Model judgment governs context-dependent decisions, while deterministic tools execute checkable operations.
Table 2: Evaluation dimensions and corresponding measurements used to assess the quality and generation efficiency of Spark-to-Paper.
MetricMeasurement
Quality
Citation validityResolved references / total references
Fabrication detectionDetected unsupported claims / injected claims
Figure editabilityEditable elements / total figure elements
Review precisionVerified issues / raised review issues
Cross-template robustnessSuccessful templates / supported templates
Efficiency
Generation costMonetary cost, tokens, wall-clock time, and deployment footprint
Figure 3: Integrity and correction in Spark-to-Paper. Deterministic gates enforce verifiable properties, Self-Review and Adversarial Review challenge semantic decisions, and surviving issues trigger revision. Experiment–critique–revision cycles are capped at seven; an unresolved trajectory becomes a failure report rather than a successful manuscript.
Figure 3: Integrity and correction in Spark-to-Paper. Deterministic gates enforce verifiable properties, Self-Review and Adversarial Review challenge semantic decisions, and surviving issues trigger revision. Experiment–critique–revision cycles are capped at seven; an unresolved trajectory becomes a failure report rather than a successful manuscript.
Table 3: Main comparison of Spark-to-Paper with human-written preprints, prior autonomous research systems, and a single-pass LLM baseline across artifact quality and generation efficiency metrics.
SystemCitation exist. (%)Fig. editability (%)Tokens (M)USDWall-clock
Human-written preprints (sampled)97.8 [94.6, 99.4]58 [44, 71]n/an/an/a
AI Scientist, released papers [13]93 (42/45)0 (0/210)n/r$10–15 (amort.)∼12 h / batch
AI Scientist-v2, workshop set [28]91 (58/64)3 (0–8)n/r∼$20–25 / attempt≤15 h / run
Agent Laboratory, released paper [20]96 (27/28)0 (0/30)n/r$2.33 (gpt-4o)∼19 min
Single-pass LLM draft (same backbone)81 (range 76–86)n/a0.11 (0.09–0.13)$0.66 (0.55–0.76)16 min (13–19)
Spark-to-Paper (full stack)99.5 [98.4, 100]96.4 [92.7, 98.6]11.9 [10.2, 13.7]$8.1 [6.9, 9.6]3.2 h [2.6, 3.9]
Figure 4: Role-aware editable figure generation. Measured results follow deterministic plotting to native vector PDF, whereas explanatory content uses a raster visual target followed by iterative HTML reconstruction and vector export; unreliable reconstruction falls back to the raster.
Figure 4: Role-aware editable figure generation. Measured results follow deterministic plotting to native vector PDF, whereas explanatory content uses a raster visual target followed by iterative HTML reconstruction and vector export; unreliable reconstruction falls back to the raster.
Table 4: Ablation study of the Spark-to-Paper quality stack, showing the contribution and incremental cost of gating, self-review, and adversarial review.
ConfigurationFabr. detection (%)Review precision (%)Δ tokens (M)Δ USD
Single-pass draft (no gates)14 (5/36) [6, 29]n/aref.ref.
Gates only69 (25/36) [53, 82]n/a+8.1 ± 0.9+5.3 ± 0.5
Gates + self-review81 (29/36) [65, 90]n/a+1.1 ± 0.2+0.6 ± 0.1
Gates + self-review + adversarial review92 (33/36) [78, 97]74 (42/57) [61, 83]+2.6 ± 0.4+1.6 ± 0.2
Figure 5: Page, reference, and figure counts for the end-to-end papers documented in the project’s own showcase materials. Self-reported by the system’s maintainers, not an independent measurement; included as existence evidence, not as a benchmark result.
Figure 5: Page, reference, and figure counts for the end-to-end papers documented in the project’s own showcase materials. Self-reported by the system’s maintainers, not an independent measurement; included as existence evidence, not as a benchmark result.
Table 5: Claim admission labels and corresponding revision actions.
LabelAction
supportedRetain with evidence-matched wording
partially-supportedNarrow the claim or request additional evidence
unsupportedRun a feasible missing experiment, weaken, or remove
contradictedRemove or report as a limitation
needs-confirmationReturn the unresolved claim for author confirmation
Figure 6: Case study on two different domain demo papers with only one short proposal as inputs, incorrect expectations are highlighted by Bold and Underline text.
Figure 6: Case study on two different domain demo papers with only one short proposal as inputs, incorrect expectations are highlighted by Bold and Underline text.
Table 6: Main persistent artifacts used by Spark-to-Paper.
ArtifactStagePurpose
blueprint.jsonPlanningPaper structure, claims, notation, experiments
template.jsonPlanningVenue specification and execution mode
refs.bibCitationVerified bibliography
claims_map.jsonCitationClaim–citation associations
sections/*.texWritingSection-level manuscript sources
figures/FigureFigures and generation records
results.facts.jsonDataGrounded quantitative evidence
main.tex/pdfAssemblyFinal manuscript project
logs/*.io.mdAll stagesStage-level input and output records
Figure 7: Paper Demo (Part 1): Screenshot for the demo paper introduction.
Figure 7: Paper Demo (Part 1): Screenshot for the demo paper introduction.

Findings

  • Across eight controlled research topics, the full system achieved 99.5% citation validity and 96.4% figure editability.
  • In a controlled ablation with injected unsupported claims, fabrication detection rose from 14% for a single-pass draft to 92% with the complete integrity and review stack.
  • Adversarial review achieved 74% precision on the issues it raised.
  • The full system used 11.9M tokens and cost $8.1 per manuscript on average, requiring 3.2 hours of wall-clock time.
Figure 8: Paper Demo (Part 2): Screenshot for the demo paper method.
Figure 8: Paper Demo (Part 2): Screenshot for the demo paper method.

Where it can be used

  • Expanding a rough research idea into a complete draft manuscript quickly.
  • Automatically updating a paper's text, tables, and figures once code and experimental results already exist.
  • Using the fabrication-detection and citation-validation checks as an auditing aid for draft manuscripts.
  • Referencing the claim-revision workflow (narrowing or moving claims to limitations) when experimental outcomes diverge from expectations.
Figure 9: Paper Demo (Part 3): Screenshot for the demo paper experiment and analysis.
Figure 9: Paper Demo (Part 3): Screenshot for the demo paper experiment and analysis.

Limits and open work

  • Evaluation covers only eight controlled research topics, so generalization to broader fields or real submission conditions is not established.
  • The comparison against other systems in Figure 1 and Table 1 is the authors' own ordinal assessment from public documentation, not a measured benchmark.
  • The page, reference, and figure counts for showcase papers in Figure 5 are self-reported by the system's maintainers, not independently measured.
  • Judging argument persuasiveness, contribution importance, or scientific appropriateness of experimental design still relies on model judgment rather than deterministic verification.
  • Semantic assessment of whether evidence supports a claim is currently performed by the model, and the accuracy of this specific judgment step is not separately measured.
Figure 10: Paper Demo (Part 4): Screenshot for the demo paper experiment and analysis.
Figure 10: Paper Demo (Part 4): Screenshot for the demo paper experiment and analysis.

Why it matters

It shows a path for merging the currently separate workflows of running experiments in code and drafting papers in another tool, by keeping everything inside the coding assistant researchers already use. It also gives a concrete, measured design for catching unsupported or fabricated claims, which matters for anyone evaluating the trustworthiness of automated research-writing tools.

Terms in this paper

  • Deterministic gate · A mechanical check with explicit pass/fail rules, such as citation validation or LaTeX compilation success
  • Self-Refutation Loop · A failure pattern where repeated experiments keep concluding the original research objective isn't supported, yet the system keeps revising the same direction instead of stopping
  • Adversarial Review · A manuscript-level stage where multiple independent review passes challenge the paper from different angles, like theoretical soundness or experimental design
  • Data-Aware Mode / Proposal Mode · Two execution modes selected depending on whether measured experimental data is already available, controlling how strictly quantitative statements must be grounded

Original abstract (English)

Turning a research idea into a complete paper requires more than text generation: the system must retrieve literature, design and execute experiments, revise claims according to evidence, produce publication-ready figures, and maintain consistency across a long generation process. We present Spark-to-Paper, an end-to-end research paper generation system implemented as thirteen composable skills inside an existing coding assistant, without requiring a separate agent platform or orchestration service. Spark-to-Paper separates model-based judgment from deterministic operations that can be directly executed and checked. It further separates experiment planning from reporting, so that required evidence is specified before results are observed and manuscript claims are revised according to measured outcomes. To improve reliability over long research trajectories, the system combines deterministic integrity checks with self-critique and bounds a failure mode we call the Self-Refutation Loop, in which repeated experiments continue to reject the original research objective. Spark-to-Paper also produces editable vector figures through programmatic plotting for experimental results and code-based reconstruction for generated method diagrams. Across eight controlled research topics, Spark-to-Paper achieves 99.5% citation validity and 96.4% figure editability. A controlled ablation increases fabrication detection from 14% for a single-pass draft to 92% with the full integrity and review stack, while adversarial review achieves 74% precision. The full system uses 11.9M tokens, costs $8.1 per manuscript, and requires 3.2 hours on average. These results show that end-to-end research paper generation can be implemented as a lightweight, composable workflow inside existing coding assistants while keeping experimental evidence central to how claims are accepted, revised, or abandoned.

Authors · Zhuoyang Qian

Read on arXiv

Latest papers

All papers →

Latest from METAL MEDIA

Figures: Zhuoyang Qian et al., arXiv:2608.11924, arxiv-nonexclusive