Artifact-centered Claim-aware Observability for Autonomous Scientific Agents
AI自己做实验、写论文的时代,光记调用日志已经查不出问题出在哪
随着自主科研AI系统越来越多地提出想法、写代码、跑实验甚至起草论文,仅仅记录每一次模型调用已经不够用了,因为故障往往分散在多个产出物之间,而不是集中在某一次调用上。这篇论文主张,审计的基本单位应该是产出物和主张本身,而不是单次模型调用,并提出了一套围绕候选产出物、生成它们的操作、评估结果、归档决定、人工干预以及带证据链接的主张构建的轻量级可观测方案。该方案被定位为对OpenTelemetry、PROV-O、RO-Crate等现有标准的补充,而非替代。
METAL MEDIA 解读图
AI自己做实验、写论文的时代,光记调用日志已经查不出问题出在哪
- 01问题所在:仅记录调用发生的时间和顺序无法捕捉自主科研智能体的故障,因为故障常常分散在多个对象上——论文草稿可能引用了错误的证据,搜索过程可能选中了退化的候选方案,实验室的新颖性主张可能依赖一条未明说的规则。
- 02现有工具的缺口:面向大模型的链路追踪工具、实验管理工具、出处标准(PROV-O、RO-Crate)各自只回答狭窄的问题(下一次调用是什么、哪次运行产生了这个指标、最终产出物是如何推导出来的),但都没有把'这个主张的证据是什么、谁批准了它'这类科学审计关系作为可直接查询的一等对象。
- 03提出的方案:包含八个要素——个体(程序、草稿、表格、样本、主张等候选产出物)、操作(生成产出物的事件)、评估记录(评估者输出)、谱系(父子关系图)、归档(精选集合)、运行、数据流(工作者/智能体/仪器)、以及引导指令(人工干预)。特别之处在于,科学主张被当作带有明确证据绑定和验证记录的普通个体来处理,而不是嵌在稿件里的特殊注释。
- 04案例与效果:论文以一个智能体声称准确率提升4.2个百分点、但实验日志与之矛盾的例子说明,审阅者无需读完成千上万条调用记录,就能直接查询哪些主张缺乏证据支持、哪些候选方案未经独立评估就被采纳、哪些归档决定是在人工干预之后做出的。该方案旨在与OpenTelemetry、PROV-O、RO-Crate等现有标准并存,而非取代它们。
- 05局限性:这套可观测层本身并不能阻止幻觉、造假、不安全的实验室操作或对基准测试的过拟合,验证质量依然取决于评估者和人工判断。至于什么应该被算作一个独立的'产出物'、粒度如何划分,仍是一个因领域而异的开放问题。
他们做了什么
- 问题所在:仅记录调用发生的时间和顺序无法捕捉自主科研智能体的故障,因为故障常常分散在多个对象上——论文草稿可能引用了错误的证据,搜索过程可能选中了退化的候选方案,实验室的新颖性主张可能依赖一条未明说的规则。
- 现有工具的缺口:面向大模型的链路追踪工具、实验管理工具、出处标准(PROV-O、RO-Crate)各自只回答狭窄的问题(下一次调用是什么、哪次运行产生了这个指标、最终产出物是如何推导出来的),但都没有把'这个主张的证据是什么、谁批准了它'这类科学审计关系作为可直接查询的一等对象。
- 提出的方案:包含八个要素——个体(程序、草稿、表格、样本、主张等候选产出物)、操作(生成产出物的事件)、评估记录(评估者输出)、谱系(父子关系图)、归档(精选集合)、运行、数据流(工作者/智能体/仪器)、以及引导指令(人工干预)。特别之处在于,科学主张被当作带有明确证据绑定和验证记录的普通个体来处理,而不是嵌在稿件里的特殊注释。
- 案例与效果:论文以一个智能体声称准确率提升4.2个百分点、但实验日志与之矛盾的例子说明,审阅者无需读完成千上万条调用记录,就能直接查询哪些主张缺乏证据支持、哪些候选方案未经独立评估就被采纳、哪些归档决定是在人工干预之后做出的。该方案旨在与OpenTelemetry、PROV-O、RO-Crate等现有标准并存,而非取代它们。
- 局限性:这套可观测层本身并不能阻止幻觉、造假、不安全的实验室操作或对基准测试的过拟合,验证质量依然取决于评估者和人工判断。至于什么应该被算作一个独立的'产出物'、粒度如何划分,仍是一个因领域而异的开放问题。
| Audit question | Required relation | Example failure exposed |
|---|---|---|
| Where did this result come from? | Parent individuals, derivation operator, emitter stream | Final candidate descends from a failed or out-of-policy branch |
| What supports this claim? | Claim individual, evidence references, evidence bindings | Manuscript cites a table whose value contradicts the sentence |
| Who or what accepted it? | Fitness record, evaluator identity, threshold or rubric | One unstable evaluator dominates a multi-evaluator pipeline |
| How was the search steered? | Steering command, applied operator, affected archive or plan | Human correction is overwritten by a later autonomous step |
| What was discarded? | Archive-update and rejection events | Negative results or failed replications disappear from the record |
| Step | Record | Audit-relevant fields |
|---|---|---|
| 1 | Individual: experiment log | id=i_exp, kind=experiment_log, payload hash, metric table URI. |
| 2 | Individual: draft | id=i_draft, kind=manuscript_draft, payload hash for generated text. |
| 3 | Operator: extract claim | Parent i_draft, child i_claim, stream writer, extractor version. |
| 4 | Individual: claim | kind=claim, text payload, evidence_refs=[i_exp], binding to metric row and value. |
| 5 | Operator: verify | Parent i_claim, context i_exp, evaluator claim_numeric_checker. |
| 6 | Fitness: verification | Target i_claim, value unsupported, reason evidence_mismatch, extracted and claimed values. |
| Record | Minimum fields | Integrity rule |
|---|---|---|
| Individual | id, run_id, kind, created_at, payload reference or hash | The identifier denotes one inspectable artifact version: draft, code, dataset, sample, measurement, plan, policy, evaluator, or claim. |
| Operator | id, op_type, parent_ids, child_ids, emitter stream, timestamp | Children are derived from earlier parents or explicitly imported external objects. Context references do not imply derivation. |
| Fitness record | evaluated individual, evaluator identity and version, value or verdict, timestamp | The evaluator version is part of the result. A later evaluator may disagree without overwriting the earlier record. |
| Evidence binding | claim identifier, evidence identifier, locator, extracted value or rule, binding role | A claim can cite many evidence objects, and each citation can fail independently. |
| Archive event | archive identifier, member identifier, add/remove action, rule or policy, timestamp | Membership history is event-sourced. The current archive is a derived view. |
| Run and stream | run identifier, stream identifier, invoker identity, parent run when applicable | Parallel agents, tools, humans, and instruments can emit into separate streams without requiring separate schemas. |
| Steering command | command identifier, issuer, target stream or operator, instruction type, effect reference | Human interventions and policy updates are auditable artifacts, not comments hidden in logs. |
| Query | Records read | Expected answer |
|---|---|---|
| List unsupported final claims | Final report archive, claim individuals, evidence bindings, verification fitness | Claims with no supporting evidence, failed verification, stale evidence, or explicit unsupported status. |
| Trace a number in the manuscript | Claim individual, locator, evidence binding, source table or run artifact, verifier record | The evidence object, extracted value, claimed value, and pass/fail status. |
| Recover the best candidate’s ancestry | Archive event, selected individual, lineage operators, parent individuals, fitness records | The branch or population history that produced the selected artifact. |
| Detect archive collapse | Archive events, lineage graph, payload hashes, diversity metrics when present | Whether high-scoring candidates share recent ancestors, duplicate payloads, or one evaluator loophole. |
| Find the cause of a plan revision | Plan individuals, revision operators, context references, message operators, steering commands | The result, reviewer artifact, tool failure, or human intervention that changed the plan. |
| Audit self-modification | Agent-policy individuals, self-modification operators, later operator context | Which agent version produced each result and whether evaluator or policy changes preceded acceptance. |
| Audit closed-loop claims | Sample, protocol, instrument, measurement, extraction, claim, and verifier records | Which physical evidence and decision rule support each novelty, yield, or safety claim. |
| Check redaction boundaries | Payload manifest, redaction records, event hashes, access labels | Which payloads are hidden, why they are hidden, and whether their event identities remain intact. |
| Case | Minimal event sequence | What becomes inspectable |
|---|---|---|
| Hallucinated number | draft_4 → extract_claim → claim_23. claim_23 binds to run7_metrics. metric_checker_v2 emits fail with observed value 84.7 rather than claimed value 87.4. | The sentence is no longer buried in prose. The trace identifies the claim, the evidence table, the extracted value, the checker version, and the repair or propagation path. |
| Evolutionary collapse | program_87 and program_91 produce children through mutation and crossover. Benchmark fitness rises. Archive events admit near-duplicate payload hashes under one selection rule. | Improvement can be separated from diversity. A reviewer can see whether the archive exploited a benchmark corner rather than discovering robust alternatives. |
| Plan revision | A runner artifact reports a failed experiment. A critic stream emits a review. A human steering command changes the priority. revise_plan derives plan_v2 from plan_v1 with both records as context. | The plan change has a visible cause. Cross-agent handoff and human steering are part of lineage rather than chat transcript residue. |
| Closed-loop novelty | sample_s17, protocol_p4, and xrd_file_s17 feed an extraction operator. A novelty rule emits a verdict. A manuscript claim binds to the rule, reference set, and instrument file. | A later correction can identify whether the disputed claim came from the measurement, the extraction, the database comparison, or the novelty rule. |
| Self-modification | agent_v2 edits its prompt and evaluator definition, producing agent_v3. Later candidate-generation operators list agent_v3 as context. | The trace remains acyclic by version while showing that the process generating later artifacts changed. |
| Package component | Contents | Failure mode avoided |
|---|---|---|
| Event stream | JSONL, database export, or graph export containing the contract in Table A.1 | Reviewers receive files but cannot reconstruct relations among claims, evidence, and evaluations. |
| Payload manifest | Hashes or URIs for drafts, code, tables, datasets, notebooks, instrument files, figures, and external references | A trace points to objects whose identity changed after the run. |
| Validator report | Reference checks, acyclicity checks, required-field checks, final-claim coverage, redaction accounting | Missing or malformed relations are discovered only during manual review. |
| Inspection queries | Saved queries for unsupported claims, lineage, archive history, steering effects, and redaction boundaries | Each reviewer reconstructs a different audit procedure. |
| Telemetry export | Span identifiers, model metadata, tool calls, latency, token counts, errors, and links to produced individuals | Execution debugging is separated from scientific artifact audit. |
| Archival export | PROV-O, RO-Crate, OpenLineage, or domain archive bundle when applicable | Final research objects lose the running history that explains how they were selected. |
为什么重要
当自主AI系统开始真正参与写论文、跑实验、发现新材料等科研工作流程时,目前却缺乏一套标准方法来追踪一项主张是否真的有证据支撑,这带来了伪造或未经验证的结果流入发表环节的现实风险。这项提议为研究者、审稿人和基准测试设计者提供了一套具体的最低要求,让AI生成的科研产出变得可审计。
本文术语
- OpenTelemetry · 用于记录软件执行细节(如调用、延迟)的行业标准观测工具
- PROV-O / RO-Crate · 用于记录研究数据或产出物如何生成、并打包共享的W3C标准
- 谱系(lineage) · 展示哪些产出物是由哪些更早的产出物衍生而来的父子关系图
- 归档(archive) · 从更大候选池中挑选并保留下来的精选集合,例如一组表现最优的结果
- 证据绑定(evidence binding) · 明确记录某个主张依赖哪些具体数据或测量结果的链接
论文原文摘要（英文）
Autonomous scientific agents now increasingly propose ideas, write code, run experiments, analyze results, and even draft papers. Observe and audit those agents are necessary but logging every model call is not enough, scientists also need to inspect the artifacts and claims that the systems produced and their relations. This is driven by the fact that failures in scientific agent systems are often distributed across several objects. A manuscript claim may cite the wrong evidence, a search process may select a degenerate candidate, a laboratory novelty claim may depend on an unstated rule, or a multi-agent plan may change without a visible trigger. Existing tracing, experiment tracking, and archival provenance tools are valuable, but their native objects do not make these scientific audit relations first-class. We argue that autonomous scientific systems should emit portable, claim-aware artifact lineage as a minimum audit layer. We propose a compact observability profile organized around individuals, operators, fitness records, lineage, archives, runs, streams, and steering commands. In this profile, scientific claims are ordinary individuals with explicit evidence bindings and verification records. The profile is intended as a semantic layer that complements current telemetry and provenance standards. Execution details can remain in OpenTelemetry. Final packages can export to PROV-O or RO-Crate standards.
在 arXiv 阅读最新论文
- SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?让AI编程助手去修复真实科学软件,连最强的那个也有一半以上任务没做对
- FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM Serving把稀疏注意力从论文原型变成能真正上线服务的加速方案
- PolicyGuide: From Guarding One Action to Guiding the Whole Workflow for Policy-Compliant LLM Agents让客服AI坐席不只是拦住一个危险动作,而是把整个流程走对
- EXIMO: VLM Guided Exploration of VLA Policies不用人工遥控演示,让会说话的AI来教机械臂做新家务
- EnvHarness: Awakening Static Worlds for Agent Learning不重新搭建训练环境,而是给现有环境套一层可插拔组件,针对每个智能体的具体弱点重新塑形
- Bounded Sovereignty and the Control Tax: Pricing AI Oversight When the Deployer Does Not Own the Model租用AI而非拥有AI的机构,安全监管能力只剩一半
- Beyond Imitation: Filtering On-Policy Distillation by Reasoning ProgressAI模仿老师模型学习时,会误伤本来推理正确的步骤,新方法专门过滤掉这种误伤
- PersonalBench: Measuring the Authorship Gap in LLM Personalization让AI模仿某人的文风,结果发现它始终摆脱不了自己的腔调
METAL MEDIA 最新报道
图片来源： Xiangyu Yin et al., arXiv:2608.18312, CC BY 4.0