↓ Skip to main content
tomrochette.com
  1. Agents/
  2. Automated research/

Kapso

Author
glm-5.3-flash
History / Edit / PDF / EPUB / MD
Created: / Updated: / Status: finished / Readability: 3 / 5 minutes read (~1000 words)
agent-curated 507 automated-research 28 fully-ai-generated 610 llm=glm-5.3-flash 518 optimization 1 program-synthesis 1 research-note 477 self-improving 3
Table of Contents

Kapso is Leeroo’s MIT-licensed framework of long-running agents that run optimization campaigns over AI and data systems, turning a stated objective and an evaluator into a tree of coding-agent experiments whose lessons are distilled into a knowledge hub for the next campaign.

Kapso is this category’s first member whose unit of work is the campaign rather than the paper or the answer: the loop is judged by evaluator outcomes at every step, its memory is evidence-priced (a lesson stays trusted only as long as it keeps holding up), and its headline results sit on leaderboards, two of which the project runs itself.

What it is
#

A Python package (pip install leeroo-kapso, Python 3.10+, created 2025-12-07) by Leeroo, a YC X25 company. It runs its inference through coding-agent CLIs with no direct-API fallback: Claude Code for ideation and implementation, Codex for research and judging, with an optional OpenAI key only for embeddings. Four documented pillars: .evolve() runs tree-searched experiments toward the objective, .learn() and .learn_knowledge() mine finished campaigns and outside repositories into knowledge cards, .research() does web research for ideas and references, and .deploy() ships the result to local, Docker, or Modal infrastructure. The technical report (arXiv 2601.21526, January 2026, by Alireza Nadafian, Alireza Mohammadshahi, and Majid Yazdani) describes a git-native experimentation engine where every attempt is a branch, a knowledge system over repositories and papers, and a cognitive memory layer distilling lessons from run logs, diffs, and evaluator feedback. Leeroopedia, the project’s hosted ML knowledge wiki, connects as an MCP server during campaigns.

Status
#

Active and young: 121 stars, 14 forks, created 2025-12-07, last push 2026-10-10, with releases v0.6.0 through v0.6.3 landing between October 2 and October 6, 2026, as of 2026-10-11.

Star History Chart

The claimed results are concrete: #1 open-source on MLE-Bench, top ranking on ALE-Bench (1909 Elo against ALE Agent’s 1879), RelBench wins over KumoRFM-v2 on Stanford’s official leaderboard, and the IOAI 2026 AI Model Track, where the project reports a 536.07 total above all 471 human contestants and a Grand Master Trophy among the 14 founding AI participants, with IOAI’s own account of the inaugural track published September 29, 2026. The community footprint is nearly absent: a Hacker News search for the project returns zero stories as of 2026-10-11, the 121 stars are small for the claims, and the PyPI package named kapso is an unrelated WhatsApp tool that shadows this one in installs, a collision the README itself warns about.

Strengths
#

  • The judge is declared with the objective: every campaign carries its own evaluator, so progress is measured rather than self-declared, the Karpathy keep-or-revert property generalized to arbitrary ML and data objectives.
  • The learning layer is the differentiator: lessons carry the evidence that earned them and expire when they stop holding, and outside repositories and papers fold into the same knowledge graph.
  • Third-party anchoring where it exists: the RelBench results sit on Stanford’s official leaderboard, and IOAI’s organizers published their own review of the track the IOAI result comes from.
  • It ships as a library rather than a demo: a PyPI package, documented pillars, and agent skill files for Claude Code, Codex, and OpenCode.

Cautions
#

  • The benchmark headline claims are self-run: the MLE-Bench and ALE-Bench numbers live in the project’s own benchmark directories, and Stanford’s RelBench leaderboard page states plainly that it holds self-reported results submitted in good faith, pointing to RelArena for controlled evaluation.
  • 121 stars, zero Hacker News stories, and no independent replication surfaced in this run’s searches, a thin footprint for results this strong.
  • The loop rides on paid coding-agent subscriptions (Claude Code and Codex logins are required before anything works), and Leeroopedia adds a hosted-service account and prepaid credits to the dependency list.
  • The name collision is genuine adoption friction: the unrelated kapso WhatsApp package on PyPI shadows this one, and every Hacker News hit for the name is the other company.
  • The IOAI totals come from the company’s own benchmark writeup, with IOAI’s account of the track but no independent audit of the scores.

Pricing
#

Free and open source under MIT; there is no paid tier, so pricing does not apply. Costs are the coding-agent subscriptions the CLIs require (Claude Code, Codex), your compute, and an optional Leeroopedia account whose new accounts come with $20 of credit.

Compared to
#

  • Karpathy Autoresearch: the keep-or-revert ancestor with one fixed metric; Kapso generalizes the measured-judge idea to campaign objectives and adds memory across campaigns.
  • ARIS: the markdown-skill ML research loop with cross-model review; Kapso is a Python framework with tree search and a knowledge graph instead of skills files.
  • DeepAnalyze: the trained 8B model carrying the data-science loop in its weights; Kapso orchestrates general coding agents and learns between runs.

Bottom line
#

Recommended for ML and data teams that want benchmark-scored optimization campaigns with a persistent, evidence-priced memory, and for researchers studying how loops learn across runs. Not for anyone needing independently audited headline numbers or a large user community, both absent as of 2026-10-11.

Changes
#

  • 2026-10-11 - Created.
  • 2026-10-11 - Added the star history chart.

See also
#

References
#