BenchLM is a solo-maintained composite leaderboard that blends public benchmark evidence and other published leaderboards into one BenchAlign score per model, and labels every ranked position by how much evidence stands behind it.
It is the only member of this category whose product is not a measurement but a provenance-labeled reading of everyone else’s measurements.
What it is #
A website, a downloadable dataset, and a query API covering 889 model profiles and 625 canonical benchmark pages, with 218 ranked entries (83 Supported, 135 Estimated) across 8 weighted categories (as of 2026-10-10). The BenchAlign score (v5.8) starts from independent external leaderboards, benchmark composites and preference boards, each weighted by uncertainty, with preference leaderboards capped at a quarter of the external part and direct benchmark results adding a correction of at most 3 points. Missing tests stay missing, generated rows never enter the boards, and the confidence page cuts the 218 into four levels: 27 Full, 60 Good, 67 Limited, 64 Estimated, with sourced-versus-generated row counts per model. Data access is unusually complete for the genre: eight static JSON downloads with a SHA-256 manifest and content versions, plus a free Data account exposing REST and MCP endpoints under a CC BY-NC 4.0 license for BenchLM’s own outputs. It is built and maintained by one person publishing as Glevd, with an email contact and an X handle as the only disclosed identities.
Status #
Active and fresh: the dataset, leaderboard, and comparison files were last updated 2026-10-10, pricing and speed files a day or two before, and the scoring policy iterates in dated steps (BenchAlign v5.8 with a September 30, 2026 retrospective). The community footprint is close to zero: an HN search finds no BenchLM story of its own as of 2026-10-11, and the one independent sighting is a June 2026 Show HN (Models Pie, 4 points) that names BenchLM as one of its three data sources. No company, funding, or team is disclosed, so treat it as a one-person project with institutional-scale coverage and no institutional ballast. Treat it as a working cross-check layer, not a source of record.
Strengths #
- The evidence labels are the genre’s best: a reader can see, per rank, whether the position rests on exact sources or on the estimator’s blend, and the confidence page counts sourced versus generated rows per model.
- The methodology publishes its own failures: three retrospective checks of 95th-percentile error worsened, and the page says so rather than burying it, which no larger rival in this category does.
- Agent-facing data without a paywall to start: hashed static JSON for offline use, plus REST and MCP on a free tier.
Cautions #
- It runs nothing itself, so every number is another board’s measurement twice removed, and the errors inherit: a distorted input leaderboard distorts the blend.
- The headline breadth overstates the depth: 889 tracked profiles reduce to 218 ranked entries, and only 83 carry exact-source support.
- The Estimated intervals carry the operator’s own warning that they do not establish confidence in a rank or future performance, and the category intervals are unvalidated after the latest prior changes.
- CC BY-NC 4.0 blocks commercial republication without a separate agreement, a sharper restriction than this category’s other open datasets.
- One pseudonymous operator, no repository, no funding disclosure, and no second pair of eyes on the pipeline.
Pricing #
The site is free to read. A free Data account covers REST and MCP queries; Pro and Research plans add saved ranking history and publisher snapshots, with no public prices. Commercial display or redistribution of BenchLM-owned outputs needs a separate agreement, so no stated price exists and no price history applies.
Compared to #
- LLM Stats: the other composite aggregator; choose LLM Stats for its API maturity and MCP tooling, BenchLM when the question is how much evidence sits behind a rank.
- Benchmark Radar: the catalog of benchmarks themselves; BenchLM consumes that kind of evidence to rank models instead.
- Artificial Analysis: first-party measurement with published weights; BenchLM is a second opinion on that measurement, not a replacement.
Bottom line #
Recommended as an evidence-reading layer over other people’s leaderboards, and for agents that want hashed, dated JSON; not as a primary source for any deployment decision, because nothing in it was measured by the site that publishes it. My disagreeable claim: the most valuable writing on BenchLM is not the leaderboard but the failed-checks paragraph in its methodology, because publishing your own broken validation is exactly the artifact every larger player in this category declines to ship.
Changes #
- 2026-10-11 - Created.
See also #
- LLM Stats - the incumbent composite aggregator BenchLM re-ranks against
- Benchmark Radar - the benchmark-discovery layer whose catalogs feed composites like this one
- Artificial Analysis - the first-party measurement layer BenchLM consumes as an input
- Trackers and Leaderboards Feature Matrix - the category comparison this note joins
- Model Selection for Coding Tasks - where a composite rank lands when a model choice gets made
References #
https://benchlm.ai/ - homepage: 889 tracked, 218 ranked, 83 verified, BenchAlign leader Claude Opus 5.5 at 86.43, last verified 2026-10-10 (fetched 200, 2026-10-11)
https://hn.algolia.com/api/v1/search?query=benchlm&tags=story - no BenchLM story of its own; the June 2026 Models Pie Show HN names BenchLM as a data source (fetched 200, 2026-10-11)