← Research

Scaling Discovery Does Not Require Forgetting: Verified Shared Archives Break the Multi-Agent Plateau on Anthropic’s Performance Take-Home

Lindell S. Cumes QWNTL Labs

PDF · 21 pages · 399 KB

The shared verified archive keeps descending after parallel and communicating search plateau.
Figure 1. The shared verified archive keeps descending after parallel and communicating search plateau. Each arm ran four Claude Opus 5.5 agents at medium effort on Anthropic’s performance take-home, with three seeds per arm launched together and all arms given the same machine, hours and effort and the same starting kernel and protected evaluator. (a) Best verified cycles so far (lower is better). Lines are seed medians, and bands span the three seeds. Every arm starts from the 147,734-cycle kernel, so the curves enter from above the axis. The horizontal dashed line is CORAL’s published 1,103 cycles with Claude Opus 4.6, a different model, and “false floor” marks the score that several CORAL-faithful agents concluded was near the floor. (b) The archive’s lead at the least favorable pairing, the best seed of the other arm minus the worst archive seed, every 15 minutes. It is positive at every checkpoint from 2.75 hours, the vertical line in both panels. Triangles mark two early values below the axis (−88 and −85).

Abstract

Tan et al. report that longer, parallel, and communicating agent searches “all … eventually plateau as the agents accumulate history” and that “the key to scaling discovery further is to forget,” which for them means discarding every agent’s session and the candidate history and restarting from the best candidate. Their experiments never reset sessions while keeping candidates and cannot tell which of the two causes the plateau. We introduce a shared verified archive that bounds each agent’s conversation and keeps every evaluator-verified attempt with its source and cited tools, where any agent can run the attempt or its tools. On Anthropic’s performance take-home, Tan et al.’s forgetting team reaches 1,077 cycles and CORAL 1,103, both with other models. Four Claude Opus 5.5 agents on the archive reach a median of 914 [913, 916] cycles over 3 seeds in 8 hours, against 948 for a CORAL-faithful team and 946 for the best of four isolated agents on matched machines. After 3 hours both baselines’ medians stall between 946 and 956 cycles. Several CORAL-faithful agents concluded that about 951 cycles was near the floor, and 7 of 12 fell into a no-op resume loop. On matched machines the archive’s scores went below that false floor. From 2.75 hours on, the archive’s worst seed stays ahead of both baselines’ best seeds. On open problems an archive team built 60 of its first 98 new bests on another agent’s attempt or tool, and in one run per arm it was ahead of every other arm at 8 hours on AHC026 and on Erdős’s minimum overlap problem and the first autocorrelation inequality.

Code and data

Per-seed anytime curves

  • evidence/analyst_budget_curve.csvTake-home best-so-far cycles per attempt for every seed of the main comparison, the original run, the added seed and the c2d-highcpu-56 rerun70 KB
  • figures/hero_data.csvFigure 1 data: the nine main-comparison take-home runs and the open-problem arms, every best-so-far step66 KB
  • evidence/curves_matched.csvOpen-problem best-so-far for every arm and problem on a 0.25-hour grid of matched working hours435 KB
  • kx/results/curves-completed.csvTake-home best-so-far curves for the model-change, mixed-family, seeded, open-weight and single-agent runs, with the agent and model of each new best18 KB
  • kx/results/analysis-completed.txtBest-so-far cycles at 15 to 480 minutes for each take-home run in curves-completed.csv, including the mixed-family, seeded, open-weight and model-change runs2.2 KB

The take-home task prompt

C₁ and Erdős constructions with their exact verifiers

  • anc/ac1-SX-A43.txt.xzon the new-constructions pageC1 construction at 3.97 working hours, 262,144 heights, bound 1.50274043592276697641…1.3 MB
  • anc/erdos-SX-A51.txton the new-constructions pageErdős minimum-overlap construction A51, 3,584 cells, bound 0.38085673888331742055…56 KB
  • anc/verifier/audits_crossfamily/exact.pyon the new-constructions pageIndependent exact checker for C1 and Erdős (GMP carry-free packed convolution, exact fractions)8.2 KB
  • anc/verifier/verify_all.pyon the new-constructions pageRuns every checker on the construction files and compares with the papers' values6.1 KB
  • anc/SHA256SUMSon the new-constructions pagesha256 of every construction and checker file40 KB
  • anc/README.txton the new-constructions pageFile formats, exact values and how to run the checkers6.1 KB

Cite

@misc{cumes2026scaling,
  title         = {{Scaling Discovery Does Not Require Forgetting: Verified Shared Archives Break the Multi-Agent Plateau on Anthropic's Performance Take-Home}},
  author        = {Cumes, Lindell S.},
  year          = {2026},
  url           = {https://qwntl.com/paper-societies-horizon}
}
Continue reading →