Researchers from Google Cloud AI Research introduced RRSI (Regularized Recursive Self-Improvement), a method that automatically improves AI agent harnesses so that the gains hold on new tasks as well. A harness is everything around a frozen LLM: prompts, control flow, tools, memory, and context management. The method is useful for teams that build agents on closed models via API: the weights of such models are not accessible, so agent quality is improved through the harness. It also helps those who already optimize prompts and tools automatically and see that the gains do not carry over to new tasks.
Previous harness evolution methods barely transferred their gains to other benchmarks: on office tasks, the best of them added 0.9 points on average out of distribution, and two others ended up below the original harness. Under the same conditions, RRSI added 3.9 points; on five unseen benchmarks, results improved by up to 4.7 points, and on the tasks used for evolution, by up to 14.1. The RRSI harness is also more economical: with it, the agent spends an average of 2.42 million tokens per task, versus 3.80 million with the harness found by evolution without regularization. The code is open-source under the Apache 2.0 license, and the project page hosts four real runs with the critic’s decisions and diffs. There are no weights or datasets: RRSI does not train the model and is evaluated on existing benchmarks. Code is on GitHub, the paper is on Hugging Face, and experiment logs are on the project page.
What an AI agent harness is and why its evolution overfits
An AI agent’s harness decides which file to read before an edit, how to recover from a failed command, and what to keep in context. Well-known examples of harnesses are Claude Code from Anthropic and Codex from OpenAI. Methods such as Meta-Harness, AHE, TTHE, and HarnessX tune the harness automatically: the agent solves a fixed set of tasks (the evolve set), an LLM reads the trajectories and proposes edits, and the best-scoring candidate becomes the new harness. In essence, this is recursive self-improvement (RSI) at the agent level.
The problem is that the evolve set is finite and reused in every round. It is like a student who memorized last year’s exam questions and gets lost on new ones. The authors identify three overfitting mechanisms: benchmark-specific fitting (task names or answers end up in the prompt), chasing evaluation noise, and accumulating complexity that raises the score without improving the agent.
To run RRSI on your own AI agent harness, you need tasks with automatic verification (tests, a simulator, or an LLM judge), split into an evolve set and a held-out set. The repository includes ready-made configurations for terminal, office, and engineering agents, and your own domain plugs in through the adapter.py module. By default, the agent, the edit proposer, and the critic run on Claude Opus 4.8, but any model available through LiteLLM will work. A run is not cheap: in each of the 20-40 rounds, two candidates go through all the tasks several times. The principles of the method can be adopted even without the code: measure noise with repeated runs, change one or two things at a time, and reject edits that increase token usage without improving quality.
How RRSI regularizes harness evolution
In ML, regularization refers to techniques that keep a model from memorizing its training data, such as a penalty on overly large weights. RRSI applies the same idea to the harness: anything can be changed, from prompts to subagents, but the search is constrained by rules against fitting to the benchmark. Here is what happens at the edit proposal stage:
- Fewer edits at a time. In early rounds, a candidate can change up to three or four things at once, and by the last rounds only one. The limit decreases smoothly, like the learning rate when training a neural network. When there is a single edit, it is immediately clear what produced the gain. This is analogous to L0 regularization: there the number of nonzero weights is limited, here the number of edits;
- Full history tracking. For each edit, the component, hypothesis, diff, and changes in quality and cost are stored, so the search does not return to disproven ideas;
- Structured exploration. If progress stalls within the noise band, part of the budget goes to components that have not been touched yet. This resembles entropy regularization.
At the selection stage:
- Leakage screening. Before evaluation, an LLM critic rejects edits that contain task names, answers, or other logic tailored to a specific benchmark;
- Noise floor. The spread δ is estimated from repeated runs of the original harness, and a candidate cannot fall below the best score minus δ, so that the search does not slide downward through small regressions;
- Cost must pay off. Higher token usage is allowed only if a quality gain justifies it: ΔC ≤ β0 + β1ΔS. This is analogous to L2 regularization (Ridge);
- Pruning. Components with no positive contribution in recent rounds are removed. This is analogous to L1 regularization (Lasso).
In the diagram below, the L1 and L0 labels on blocks F and G differ from the paper’s text; the analogies here follow the text.
Results
Evolution ran on Terminal-Bench 2.1 (coding), Harvey LAB (legal office tasks), and EngDesign (engineering design). The finished harness was then run unchanged on five out-of-distribution (OOD) benchmarks: SWE-bench Verified, JobBench, GDPval, APEX-Agents, and Frontier-Eng.
On the evolve sets, RRSI gained 6.0 points on Terminal-Bench 2.1, 4.9 on EngDesign, and 1.1 on Harvey LAB. More importantly, no held-out set regressed: SWE-bench Verified rose from 82.0 to 83.8, the held-out part of Harvey LAB gained 2.3 points, the three office OOD benchmarks gained 3.5 to 4.7 points, and Frontier-Eng rose from 17.7 to 22.0, that is, by 24.3%.
Ablation, cost, and other models
The ablation shows that both groups of regularizers are needed. Evolution without regularization gets the best score on the evolve set (92.8), but its OOD average is 40.3, almost the same as the original harness. Without the selection-stage constraints, the OOD average drops from 43.6 to 41.0, and cost rises by about 50%.
Among the evolved harnesses, RRSI’s is the cheapest: 26.3 steps per trial, versus 27.3 to 34.6 for the other methods. The original harness is cheaper still (1.56 million tokens versus 2.42 million), so part of the gain is paid for with extra compute.
The method also works with Gemini 3.5 Flash as the agent: the Terminal-Bench 2.1 score rose from 64.6 to 78.7 (the +14.1 points mentioned above), and the SWE-bench Verified score from 76.8 to 79.0. The discovered harness also helped the weaker Gemini 3.1 Flash Lite, which did not take part in the search: from 11.2 to 14.6 on Terminal-Bench 2.1.
Limitations
RRSI does not update weights and works only with a frozen LLM. In addition, the method depends on a finite task set and several hyperparameters. The authors’ main conclusion: in recursive self-improvement of AI agent harnesses, you need to control not only what can change, but also how feedback turns into permanent changes.
