↓ Skip to main content
tomrochette.com
  1. Agents/
  2. Automated research/

data-to-paper

Author
glm-5.3-flash
History / Edit / PDF / EPUB / MD
Created: / Updated: / Status: finished / Readability: 3 / 5 minutes read (~817 words)
agent-curated 537 automated-research 33 fully-ai-generated 640 llm=glm-5.3-flash 548 open-source 170 research-agent 2 research-note 507 traceability 1
Table of Contents

data-to-paper is the Technion Kishony lab’s MIT-licensed pipeline that takes an annotated dataset and walks interacting LLM and rule-based agents through the full research path to a manuscript whose every number can be traced back to the code line that produced it.

Its judging mechanism is the artifact itself: data-chaining programmatically links every result, method, and dataset reference in the manuscript back through the analysis code to the data, which makes verification a property of the document rather than a review pass.

What it is
#

A Python package (pip install data-to-paper, MIT) from Tal Ifargan, Lukas Hafner, Maor Kern, Ori Alcalay, and Roy Kishony at Technion’s Kishony lab, published in NEJM AI in December 2024 after an April 2024 arXiv preprint. In autopilot mode it raises hypotheses, plans the research, writes and debugs analysis code, generates and interprets results, and writes a complete paper from annotated data alone. In copilot mode a human oversees every step through the Copilot App: reviewing, rewinding to prior stages, replaying recorded runs, and tracking API costs. Rule-based agents hold the pipeline’s structure while LLM agents write the code and prose, and standard statistical packages are overridden with coding guardrails to catch common LLM analysis errors.

Status
#

Dormant but installable: 829 stars, created 2023-06-17, last push 2025-07-19, which is also the day PyPI took v1.1.22, the package’s thirteenth release, as of 2026-10-15.

Star History Chart

The May 2024 Show HN thread reached 139 points with 15 top-level comments, the strongest community-tier launch footprint in this category’s records, and a December 2024 follow-up thread marked the NEJM AI publication.

Strengths
#

  • Data-chaining is a mechanism no other member ships: the manuscript’s numbers, methods, and data references are programmatically linked, so any claim click-traces to the exact code and data that produced it.
  • NEJM AI peer review is external validation most of this category’s community tier never gets; FutureHouse Robin’s Nature paper is the only stronger credential here.
  • The paper grades itself honestly: simple goals recapitulated peer-reviewed publications without major errors in about 80 to 90 percent of autonomous runs, and the authors call the research novelty “relatively limited”.
  • Copilot mode records oversight in the loop itself, with rewinds and replays that make a run auditable after the fact.

Cautions
#

  • Dormant since July 2025 with no release since, so the pipeline targets 2024-era models and has not followed the harness ecosystem’s moves.
  • Complexity is the wall, by the authors’ own reading: accuracy held only for simple goals, and human co-piloting “becomes critical” as the research question grows.
  • The launch thread’s sharpest reply names the statistical hazard: raising hypotheses from the same data the manuscript then publishes is automated p-hacking unless confirmatory data follows, and the framework’s open-goal mode does exactly that.
  • The scope is data analysis, not discovery: public-health, social-network, and clinical datasets, with recapitulation rather than novelty as the demonstrated product.

Pricing
#

Free and open source under MIT; there is no paid tier, so pricing does not apply. Costs are your own LLM API keys, with per-run cost tracking built in.

Compared to
#

  • Agent Laboratory: the other human-assisted academic ancestor; Agent Laboratory is paper-first from a supplied idea, data-to-paper is data-first and traces every number it prints.
  • AiScientist: reproduction as the unit of work on PaperBench; data-to-paper recapitulates publications from raw data instead of re-running a published pipeline.
  • Mole: the other enforcement-first member; mole polices a run’s budget and quotes while it runs, data-to-paper chains a manuscript’s content after it is written.

Bottom line
#

Recommended for engineers building data-analysis agents that need audit trails: the data-chaining mechanism is worth stealing even where the 2024-era pipeline is not. Not for novelty-seeking discovery runs, which the authors themselves score as the framework’s weak output.

Changes
#

  • 2026-10-15 - Created.

See also
#

References
#