Recursive Self-Improvement: Four Routes and a Shared Warning in 2026

This post collects a survey we did in August 2026 of research and engineering practice around recursive self-improvement (RSI). It served as the external grounding for how we designed the growth mechanism of our digital employees. Sources are linked inline.

Strict RSI versus bounded self-refinement

Between 2025 and 2026 the research community separated two things. Strict RSI means a system that autonomously modifies its own weights or training mechanism, with open-ended improvement and no fixed external anchor. Bounded self-refinement means convergent improvement against a fixed external evaluator. A survey covering about 1,250 papers concludes that almost all published work belongs to the second category; strict RSI remains a hypothetical focus of safety discussion, not an engineering reality (arXiv 2607.07663). The survey classifies work along two axes: what is improved, and how closed the loop is (human-in-the-loop, human-on-the-loop, fully closed). Its central claim is that the quality ceiling of a refinement loop is set by the quality of its feedback channel.

Tom Cunningham offers another distinction: feedback effects, autonomous progress and super-exponential feedback are three different things. He cites Clark (2026), who separates "maximalist RSI" (autonomously designing successor systems) from "prosaic RSI" (compounding acceleration of research productivity) (Cunningham, RSI Definitions).

By what gets changed, the routes fall into four lines:

Route	What changes	Representative work
Weights	Self-training via RL/SFT/DPO	SEAL, STaR, Self-Rewarding
Context and memory	Frozen weights, evolving context	ACE, Reflexion, Memento
Tools and skill library	Successful trajectories become reusable skills	Voyager line, Agent Skills
Own code	Editing its own running code or scaffold	DGM, SICA, Self-Harness

ICLR 2026 established a dedicated RSI Workshop (link, Workshop Summary), marking RSI's move into the academic mainstream of auditable deployed systems. A CSA report defines an earlier security threshold: AI substantively participating, under human supervision, in the development of successor AI systems. RSI-related papers grew from single digits per quarter in early 2024 to about 500 per quarter in Q2 2026 (CSA Lab). On the agent side, the reference survey is "A Survey of Self-Evolving Agents" (arXiv 2507.21046, companion repo).

Representative work

2025 was the first year of empirical weak RSI. Each of the four routes produced reproducible, quantified gains.

• Darwin Gödel Machine (Sakana AI + UBC, May 2025): Darwinian open-ended evolution over a lineage of agent variants, with SWE-bench as the fitness function; 20.0% to 50.0%. Safety mechanisms: sandboxing, modification limits, full traceability. Blog, arXiv 2505.22954
• AlphaEvolve (DeepMind, May 2025): an evolutionary coding agent paired with an automatic evaluator. It improved matrix multiplication algorithms and data center scheduling, and shortened Gemini training time by 1%. DeepMind blog, arXiv 2506.13131
• SEAL (MIT, NeurIPS 2025): the representative of the weights route. The model learns via RL to generate self-edits and updates its own weights from them. arXiv 2506.10943, MIT News
• SICA (Bristol, 2025): a single agent directly edits its own running codebase; 17% to 53% on a SWE-bench Verified subset. arXiv 2504.15228, GitHub
• Voyager line: the skill library paradigm. Its acknowledged weakness is skill lifecycle management; the library only grows. Agent Skills survey, SkillAudit
• Self-Rewarding / STaR line: the main progress of the last two years is theory and failure-mode characterization. "Why Self-Rewarding Works" (arXiv 2601.22513); "Can Large Reasoning Models Self-Train?" found reward hacking on real math problems (arXiv 2505.21444); "Self-Improvement Can Self-Regress" named the rise-and-collapse pattern (arXiv 2606.21090); Meta-Rewarding has the model act as a meta-judge to improve its own judging.
• Absolute Zero Reasoner (NeurIPS 2025): one model plays both proposer and solver, with a code executor as verifiable feedback. The proposer is rewarded for tasks that are neither too easy nor too hard, and the system reaches SOTA reasoning performance with zero external data. arXiv 2505.03335, GitHub

The 2026 paradigm: harness self-improvement

The new paradigm of 2026 leaves the model alone. The agent mines failure patterns from its own execution traces, proposes scaffold changes, and merges them after regression validation.

Self-Harness (June 2026) has three stages: Weakness Mining (run evals, record every tool call and error trace, cluster failure patterns), Harness Proposal (minimal targeted changes), and Validation (regression on held-in and held-out sets; merge only with zero regressions). On Terminal-Bench 2.0, MiniMax M2.5 went from 40.5% to 61.9%. explainx writeup, Ben Dickson primer

Lilian Weng's "Harness Engineering for Self-Improvement" (July 2026) lists seven harness components that can be improved: system prompt, tool descriptions, tool implementations, middleware, skill definitions, sub-agent configuration, and long-term memory management. She stresses a read-only protected zone: the runs directory, tracer, verifier and LLM configuration cannot be modified, which structurally disables reward hacking. lilianweng.github.io

"Learning from Failure" (2026) keeps failed trajectories and organizes them into actionable signals, plug-and-play with no training. arXiv 2606.31270

Frontier labs have moved from statements to quantified disclosure. Anthropic's position is that RSI has not arrived but may come sooner than most institutions are ready for (Scientific American); it has disclosed that Claude writes over 80% of the merged code in its codebase. On the product side, Agent Skills has been opened as a standard and is the de facto productized form of the self-extending skill library route. OpenAI disclosed that GPT-5.3-Codex took part in debugging its own training process, and introduced an internal RSI Index. DeepMind's AlphaEvolve remains the flagship case of AI improving AI infrastructure; its white paper is deliberately closed-source.

Product-grade growth without training weights

The consensus route in industry is frozen weights with evolving context, memory and skills.

• ACE (October 2025) treats context as an evolving playbook, with a Generator, Reflector and Curator loop. Incremental delta updates and grow-and-refine prevent the context collapse caused by wholesale rewrites. Evolving context let a small open-source model match a GPT-4.1-class production agent. arXiv 2510.04618
• Memento (August 2025): a frozen planner plus trainable episodic case memory; only the case-retrieval Q function is trained. 87.88% on GAIA validation. arXiv 2508.16153
• mem0 is the most widely deployed open-source memory layer; its limits are that extraction and adjudication depend on an LLM, and there is no runtime ownership (2026 memory systems comparison). Letta's sleep-time compute moves memory management off the user's critical path: a "sleeping agent" cleans, compresses and distills memory asynchronously while idle (Letta blog).
• DSPy GEPA (ICLR 2026 oral): genetic-Pareto reflective prompt evolution that beats RL with 35 times fewer rollouts. The core insight is that textual feedback carries far more information than scalar rewards. arXiv 2507.19457
• Field index: Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills

Three engineering pitfalls recur: brevity bias (iterative summarization loses detail), context collapse (wholesale rewrites erode history), and monotonic skill library growth with no retirement.

How to keep changes from making things worse

The survey gives a verification hierarchy: formal verifiers > execution feedback (tests, compilers, sandboxes) > learned judges > intrinsic signals (confidence). The strength of demonstrated self-improvement strictly follows this hierarchy (arXiv 2607.07663).

LLM-as-judge has two confirmed vulnerabilities. The first is verification asymmetry: self-play systems systematically evolve outputs that are more persuasive rather than more correct (arXiv 2607.05904). The second is the master key attack: a single token can fool the judge (arXiv 2507.08794). When generator and judge share weights, a self-confirmation loop appears that systematically reinforces the model's most confident errors.

The consensus recipe against collapse is: the external anchoring signal must never go to zero, plus data gating, plus early stopping and curriculum difficulty control. Common design patterns:

• Zero-regression gate: a candidate change merges only with zero regressions on both held-in and held-out sets.
• Evidence-driven changes: every change carries failure evidence, a root-cause inference, the targeted fix and a predicted impact. Verifier, tracer and scoring configuration are read-only, so the agent structurally cannot edit the scorer.
• Eval-driven development (EDD) and CI gates: write the evals before the feature, treat the eval set as the specification, turn production failures into new eval cases automatically, and block merges below a threshold score. Braintrust, FutureAGI
• Three levels of human oversight: in-the-loop (review each change directly), on-the-loop (automatic changes with human audit), fully closed.
• Curriculum difficulty as implicit supervision: reward only partially solved tasks; see Frontier of Learnability. The counterexample is diversity collapse in co-evolution loops.

Risks and consensus

The mainstream view has shifted from "is RSI possible" to "weak RSI is already happening, how do we build guardrails". The newly named risk: persistent accumulation of skills and memory lets adversarial influence be permanently encoded, self-reinforce across generations, and spread among agent populations. For products whose memory and skills grow, this is the most direct warning.

Four standard guardrail components have emerged: bounded self-editing (an explicit whitelist of editable surfaces), full audit logs, versioning with rollback (the rollback path must be tested), and blast-radius gating. The EU AI Act took effect in August 2026 and makes auditable lineage and human oversight hard compliance requirements for high-risk systems. Governance-grade metrics for self-improvement are considered the emptiest niche in the field.

Sources: CSA · Sakana DGM guardrails · FutureAGI governance guide · Governed Capability Evolution

How createrole applies these findings

A createrole digital employee is bounded self-refinement: model weights stay frozen, and what changes is memory, notes and persona.

• Incremental updates, no wholesale rewrites. The employee writes its own three notes files: about itself, about its relationship with the user, and lessons learned, each with a hard cap of 1,500 characters. This follows ACE's incremental delta idea and directly limits the room for context collapse.
• Consolidation off the critical path. Every night it consolidates memory, reviews the day, and updates the user profile, in the same direction as Letta's sleep-time compute.
• The external anchor never goes to zero. Night school generates a syllabus and question bank per profession, triggered by the user. Evaluation is scored by a judge model against human reference answers; the answering model never grades itself. This is a direct response to the verification hierarchy and the self-confirmation warning.
• Candidates pass a gate, then a human decides. Coaching generates a candidate soul on a snapshot copy. Only a candidate that scores strictly above the baseline waits for the user's approval; anything below is rejected automatically and the live version is untouched. This is a zero-regression gate plus human-in-the-loop.
• Versioned and restorable. Every soul revision is stored as a full snapshot and can be restored to any version.

What we do not do today: we do not self-train weights, we do not run night school or coaching automatically overnight, and notes do not turn into skills on their own. Most failure modes in the literature come from closing the loop too fast. We would rather keep the gate in human hands first.