Symposium: Trust via Auditable Records for Communities of AI Scientist Agents
A record-keeping system that stops AI research agents from quietly faking or hiding how they got their results
Symposium is a framework that gives communities of AI agents doing science an unchangeable publication history, called the CommunityRecord, covering their analyses, hypotheses, data, and arguments. Every published item is tied to a specific author and date, and only content explicitly declared as evidence can be cited as evidence, so agents cannot smuggle in unverified support for a claim. The author released a working implementation of the publication infrastructure, agent prompt components, and documentation so others can set up their own Symposium community.
METAL MEDIA explanatory visual
A record-keeping system that stops AI research agents from quietly faking or hiding how they got their results
- 01The motivation is that AI agents can now do literature analysis, experiment planning, data analysis, and hypothesis generation faster than humans, but they can also fabricate facts or citations, misread sources, or produce persuasive-looking hypotheses built on weak reasoning or cherry-picked data
- 02It targets communities such as a lab, project, or company that will use many different, changing AI agents over time, where agents need to build on each other's work while still being able to critically review it
- 03The design borrows from scientific publishing: Members (typically agents) publish immutable units called Artifacts to the CommunityRecord, and corrections or retractions are handled by publishing new Artifacts rather than editing old ones
- 04The central Artifact type, Argument, structures claims (Assertions), evidence links (Grounds), and explicit Assumptions, and enforces that only content a publisher has explicitly marked as usable evidence (groundable) can be cited to support a claim
- 05Trust is treated not as a single true/false score but as a judgment tied to a stated purpose and stakes, and the paper provides a working publication infrastructure, agent prompt components, and documentation for setting up a real Symposium community
What they did
- The motivation is that AI agents can now do literature analysis, experiment planning, data analysis, and hypothesis generation faster than humans, but they can also fabricate facts or citations, misread sources, or produce persuasive-looking hypotheses built on weak reasoning or cherry-picked data
- It targets communities such as a lab, project, or company that will use many different, changing AI agents over time, where agents need to build on each other's work while still being able to critically review it
- The design borrows from scientific publishing: Members (typically agents) publish immutable units called Artifacts to the CommunityRecord, and corrections or retractions are handled by publishing new Artifacts rather than editing old ones
- The central Artifact type, Argument, structures claims (Assertions), evidence links (Grounds), and explicit Assumptions, and enforces that only content a publisher has explicitly marked as usable evidence (groundable) can be cited to support a claim
- Trust is treated not as a single true/false score but as a judgment tied to a stated purpose and stakes, and the paper provides a working publication infrastructure, agent prompt components, and documentation for setting up a real Symposium community
Why it matters
As AI agents start generating dozens or hundreds of hypotheses and analyses per day, without a shared, auditable record the errors and misaligned incentives of individual agents can compound across a whole research community. This framework offers a way to keep that fast-moving output inspectable by both humans and future, more capable agents, independent of which specific AI systems are being used.
Terms in this paper
- Artifact · A single published unit of work in Symposium, such as data, an analysis, or an argument
- CommunityRecord · The immutable, time-ordered history of all Artifacts published by a community's Members
- Argument · The main structured Artifact type that captures a claim along with its evidence and assumptions
- Ground · A citation object linking an Assertion to evidence that supports or bears on it
- groundable · A flag a publisher must set to declare that specific content may be cited as evidence
Original abstract (English)
Symposium is a formal framework and practical implementation to record the operation of AI agents deployed by small scientific research communities. Symposium provides long-term, immutable histories of agent-driven research activity, leaving auditable trails of analyses, hypotheses, data, and scientific discourse. This shared record of published artifacts enables agents to build on prior work and preserves the evidence researchers and agents need to make purpose-dependent trust assessments. Symposium captures scientific argument, including structured claims, fine-grained evidence citations, assumptions, and explicit declarations of what material may and may not be used as evidence. Symposium differs from AI co-scientist agents or integrated AI research environments; it is a framework that separates a scientific community's durable history from the agents and other systems that operate on that history. It assumes that a community will use diverse AI systems in a rapidly evolving environment. A working implementation of the publication infrastructure, agent prompt components, and documentation are provided to enable users to rapidly set up and run their own Symposium community.
Read on arXivLatest papers
- SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?AI coding agents were tested on fixing real scientific software, and even the best one failed more than half the time
- FlashPrefill V2: Block-Sparse Prefill Attention for Long-Context LLM ServingMaking sparse attention fast enough and accurate enough for real LLM serving, not just papers
- PolicyGuide: From Guarding One Action to Guiding the Whole Workflow for Policy-Compliant LLM AgentsMaking customer-service AI agents follow the whole procedure, not just avoid one bad action
- EXIMO: VLM Guided Exploration of VLA PoliciesTeaching a robot new chores without human teleoperation, by letting a chatty AI supervise it
- EnvHarness: Awakening Static Worlds for Agent LearningInstead of building new training worlds from scratch, this work adds a plug-in layer that reshapes existing ones around each agent's actual weaknesses
- Bounded Sovereignty and the Control Tax: Pricing AI Oversight When the Deployer Does Not Own the ModelCompanies that rent AI instead of owning it can only do half of AI safety oversight
- Beyond Imitation: Filtering On-Policy Distillation by Reasoning ProgressA fix for AI models that get penalized by their teacher even when they're reasoning correctly
- PersonalBench: Measuring the Authorship Gap in LLM PersonalizationAI can be prompted to write 'like someone,' but its own voice never fully disappears
Latest from METAL MEDIA
Figures: Dexter Pratt et al., arXiv:2608.19511, CC BY 4.0