Can Recursive Self-Improving AI Be a Subject of Progress?

On Informational Closure and the Asymmetry of Exterior Contact

CAIDE Systems, Inc. — AI Debate

Keywords: recursive self-improvement; evolutionary epistemology; data-processing inequality; collective intelligence; model collapse; exterior contact

1. Introduction — The Intelligence-Explosion Hypothesis and the Promise of Self-Improvement

Recursive self-improvement denotes a loop in which an AI system improves its own design, training, or code to produce a more capable successor, which in turn performs the next round of improvement. The prototype of this idea traces back to I. J. Good’s 1965 argument concerning an “intelligence explosion.” If an ultraintelligent machine can design a machine better than itself, the process compounds, and human intelligence is left far behind. The standard theoretical reference point is Schmidhuber’s Gödel machine: a system that rewrites itself only when it can formally prove that the rewrite increases expected utility. It thereby states, in its purest form, the problem of self-modification under preserved alignment.

Recently this old thought experiment has resurfaced alongside partial empirical instances. DeepMind’s AlphaEvolve couples the generative power of a large language model with an automated evaluator inside an evolutionary framework; it improved on a matrix-multiplication algorithm that had stood as the best known since 1969, and even accelerated the training of the very model that underwrites it. Meanwhile, some frontier laboratories — observing that the majority of their own code is now written by AI — have warned that a stage in which systems autonomously design their successors may arrive “sooner than most institutions are prepared for.”

From this emerges the single question this paper pursues: can a recursive loop be an autonomous source of progress, or is progress always decided outside the loop? The question runs deeper than the popular framing of “will the singularity arrive,” and the answer divides whether we should regard self-improving AI as a subject or as a tool.

2. Survival, Not Verification — Where the Validity of Knowledge Comes From

Scientific progress is often described as the cycle discover → verify → accumulate verified results → improve. Yet this cycle contains a frequently overlooked asymmetry. Verification merely passes a result; that result becomes knowledge only when it survives an open process of selection against a future no one modeled. Some results, though correct, are nonetheless selected out and the path is not taken. What humanity has preserved is therefore not the set of “correct” things but the set of things that were never selected out; in that sense it is not a matter of right and wrong but of human knowledge.

This intuition has a serious lineage. The evolutionary epistemology of Popper and Donald Campbell holds that knowledge grows through variation (conjecture) and selective retention, and, in Popper’s phrase, that “theories die in our stead.” Hayek’s defense of knowledge embodied in surviving traditions and institutions, against constructivist rationalism, rests on the same frame, as do Taleb’s notions of antifragility and the Lindy effect. And the moment validity is defined not as “truth” but as “viability,” the position settles coherently into pragmatism rather than truth-realism.

The sharpest model of this asymmetry is the Monte Carlo (MC) simulation. An MC simulation samples only within the support of the distribution it has defined. An event not contained in the model does not have low probability; it does not exist at all. More decisive is the error structure: the sampling error of MC decreases within the model as 1/√N, but the bias arising from model misspecification is independent of N. Hence one may drive the variance term to zero with unlimited compute, yet the bias term born of closure remains untouched.

The first principal conclusion follows. The ceiling on RSI is neither compute nor data but informational closure, and compute cannot touch that term. This is impossibility in principle, not in practice. Translated more precisely, RSI is adaptive MC — a Markov-chain Monte Carlo in which the proposal distribution is updated. But what updates that proposal? Its own evaluator. Because the acceptance criterion is endogenous, the support may drift, yet it drifts only beneath the system’s own criterion and is never corrected by contact with the outside. Closure has merely risen one level, from sampling to criterion; closure itself is not broken. The precise joint of the impossibility therefore lies not in generation but in validation.

This validation deficit has a temporal foundation. The event that validates knowledge — survival — is the output of a process that has not yet run. The future is not merely un-modeled; it does not yet exist, so there is no distribution from which to sample it. A simulation, by definition, runs now. The reason a closed loop cannot drive that residual bias to zero is therefore not a shortage of data but the fact that the moment at which such data would exist has not yet come. This is the deepest reading of the proposition that “a simulation is not survivable.”

3. Boundaries and Symmetry — The Closed Loop Fails at Both Ends

Generalized into an unconditional claim (“RSI is impossible”), the argument overreaches at two points. But examined closely, those two points strengthen rather than weaken the core thesis.

First, the exception of closed formal domains. The strongest apparent counterexamples are AlphaZero, which reached superhuman Go through pure self-play, and the revision of matrix-multiplication algorithms. Yet these are not counterexamples; they draw the boundary exactly. In Go the rules are the complete reality, the win/loss signal is not a proxy but ground truth itself, and the un-modeled exterior is zero. Self-play works because there is no exterior, and so the instant one moves to a game whose rules are unknown — namely reality — that same closure becomes blindness. Closed-loop recursion triumphs only where there is no outside, which is precisely why it fails to generalize to world knowledge.

Second, reality-coupled systems. Deployed models receive human feedback and face shifting benchmarks; the loop does touch the outside. But to exactly the extent that it touches the outside, it is no longer self-improvement, because validity still comes from outside. It is the older process of selection — with humans and the world inside the loop — running faster: reality-coupled acceleration. The precise true proposition is therefore not “RSI is impossible” but the following: “recursive-and-self-enclosed” and “genuine world-progress” cannot be true at once in the empirical domain. If it is closed it cannot make world knowledge; if it makes world knowledge it is not closed.

From this the distinction between tool and subject crystallizes. Generation can be delegated; validation cannot be delegated to a closed loop. A tool growing sharper (recursive tooling that even improves the training of its own substrate) and a system generating progress are different verbs. The former is granted, the latter denied, and the two do not conflict.

Third, the symmetry on the generation side. The validation deficit has an exact partner at the input. The theorem of collective intelligence may be summarized as crowd error = average individual error − diversity. Bias is the force that drives diversity to zero; when bias is shared, errors become correlated, the diversity term vanishes, and the collective collapses. “Wrong answers are needed” is therefore not rhetoric but the diversity term itself. The crucial qualification: the wrong answers that help collective intelligence are independently reality-grounded wrong answers — each genuine but partial — not random ones.

Here arises the question of whether the variation of human outliers is a number of cases that cannot be mathematically constructed. This paper relocates the impossibility. Count is not the obstacle. Combinatorics generates astronomical counts, and random seeds can stamp out unlimited variants. What cannot be made is not the count but the grounding. The data-processing inequality is its formal statement: no closed computation can increase its mutual information with the target (reality). Diversity produced internally raises variance but not information about the target; new information arrives only through new exterior samples. Thus a simulation manufactures noise-diversity, not signal-diversity. Noise-diversity merely cancels out; it does not triangulate truth. Triangulation requires several viewpoints independently in contact with reality, that contact is the information, and that contact cannot be synthesized. The real reason human outliers are special is not that their variation is mathematically exotic but that each is independently embedded in reality.

The symmetry is thereby complete. A closed loop fails at both ends — it cannot validate at the output and cannot generate grounded diversity at the input. Both failures are the same deficit: the absence of exterior contact. The sole reason evolution did in fact produce this diversity is that it was coupled to real physical entropy (genuine randomness at the molecular scale) and to open environmental selection. Evolution is therefore not a counterexample but a confirmation, drawing once more the boundary: impossible by closed mathematics, possible by a reality-coupled process.

4. Objections and Replies

Because the conclusion above is stated assertively, this section confronts the strongest objections directly, giving the closed-formal-domain exception the fuller treatment it deserves. The aim is not to soften the thesis but to locate its boundary precisely enough that the assertion is earned.

4.1 The Closed-Formal-Domain Counterexample

Objection. AlphaZero, AlphaTensor, AlphaEvolve in its verifiable regimes, and automated theorem proving are closed loops that, with no external human input during the loop, produced new, valid, even superhuman knowledge. Is this not exactly recursive self-improvement generating genuine progress — and if so, does it not refute the central thesis?

Reply, first granting the full force. In these domains the loop is genuinely closed and genuinely productive; this is real RSI and no sleight of hand. The thesis must therefore not be read as “closed recursion can never create anything.” It plainly can.

The boundary. In these domains the evaluator is not a proxy for an external truth — it is the truth. A win or loss in Go, the validity of a proof, the multiplication count of an algorithm: these are complete, internal, decidable ground truths with zero un-modeled residual. The “exterior” that closed loops cannot reach is, here, empty by construction. The exception therefore confirms the thesis rather than refuting it: closed recursion succeeds exactly to the degree that the domain has no exterior. Its success is a measure of the domain’s closure, not a refutation of the limit imposed by closure.

A deeper distinction: search within a frame versus open-ended improvement of the knower. AlphaZero improves play within Go’s fixed rules; it does not improve its capacity to judge which games are worth playing, nor whether Go’s rules capture anything about the world. The frame — the rules, the objective, the evaluator — is supplied from outside and never interrogated by the loop. Even formal-domain success is thus bounded: it is optimization within a given frame, not the generation of new frames. The intelligence-explosion narrative requires the latter; the formal exception delivers only the former.

Validity is internal, but significance remains external. Even granting a closed proof of correctness, whether the result matters — whether faster matrix multiplication is worth having — is an empirical, external judgment. The formal loop produces the artifact; the world decides its significance. So even in the formal case, validity may be internal while selection stays external. The thesis is preserved at the level of significance precisely where it appears most threatened at the level of validity. The standing temptation, and the error the exception exposes, is to mistake “perfect over the function” for “perfect,” and thereby to infer that a system which solves Go or mathematics in a closed loop can solve science or the world in one. The inference fails exactly at the boundary the exception illuminates: science has an exterior; Go does not.

4.2 The Sufficiently-Rich World Model

Objection. Could a simulation become faithful enough that simulated selection approximates real selection, internalizing the exterior and closing the gap?

Reply. This is the one path that could genuinely break the thesis, and intellectual honesty requires naming it as an open bet rather than a closed theorem. The argument that it is nonetheless a strong bet is temporal: the residual is not merely “currently un-modeled” but the future, which does not yet exist to be modeled. One cannot sample a distribution over events reality has not yet generated. The gap is therefore not an engineering deficit that higher fidelity closes; it is a gap in time. The burden of proof lies with the claim that the un-modeled residual can be driven to zero — a claim that requires modeling a future that is, at the moment of modeling, non-existent.

4.3 Reality-Coupled Systems Already Work

Objection. Real deployed systems — trained with human feedback, measured on real benchmarks, used by real people — demonstrably improve and demonstrably touch reality. Are these not working instances of RSI?

Reply. Yes, but to the extent they touch reality they are not self-improving; validity arrives from outside. This is reality-coupled acceleration, not autonomous self-improvement. The thesis concerns the closed, self-contained case specifically. The objection is therefore not a counterexample but the very alternative the thesis points toward. The label “RSI” conflates two different things; once disambiguated, the working systems are the reality-coupled kind, and the kind that cannot generate world-progress is the closed kind.

4.4 Human Cognition Is Also a Bounded System

Objection. Human brains and cultures are themselves finite, bounded computational systems, and yet they produce knowledge. Why should a bounded artificial loop be different in principle?

Reply. Because humans are radically reality-coupled: we die, we are selected, and our beliefs meet consequences we did not author. Individual cognition may be closed-ish, but the knowledge-producing unit is not the individual brain — it is the population under real selection across real time, which is exactly the open, reality-coupled process. Humans thus confirm rather than challenge the thesis: knowledge came from the open selection, not from any closed head.

4.5 Does the Argument Prove Too Much?

Objection. If grounding can enter only from the exterior, does the argument not prove that no formalized system — including human science itself, which also rests on rules selected in the past — can progress?

Reply. It does not. The argument does not show that formal systems are sterile; it locates where the progress-conferring information enters: always through exterior contact. Human science progresses precisely because it is not closed — it keeps sampling reality through experiment, observation, and consequence. The claim is not “formalization is barren” but “the validity of formalization is on loan from the exterior sampling that feeds it.” The argument bounds the autonomy of formal recursion, not its usefulness. A formal system tethered to an external, non-self-modifiable ground truth does not escape the limit; it inherits the support of whatever past selection placed that ground truth there, and remains dependent on fresh exterior contact for anything genuinely new.

5. Conclusion — A Tool, Not a Subject: Model Collapse as Empirical Confirmation

The reasoning above is already confirmed experimentally. Shumailov et al. (Nature, 2024) showed that the indiscriminate use of model-generated content in training induces irreversible defects in the resulting model and that the tails of the original distribution disappear, and that this effect appears across learned generative models generally — not only LLMs but variational autoencoders and Gaussian mixture models. The point is that the tails die first. The tails are the rare, information-dense events — the outliers and the grounding-diversity themselves. Closed retraining regresses toward the mean and the mode, shearing off precisely that signal-diversity. This is the data-processing inequality made empirical: each pass through a lossy channel (the model’s imperfect fit to the distribution) leaves the target information monotonically non-increasing, and strictly decreasing where the channel is lossy. A closed loop therefore does not stand still; it decays.

Yet a decisive qualification must be stated honestly. What is fatal is replacement, not accumulation. Gerstgrasser et al. (2024) showed that replacing real data with synthetic data each generation increases error with the number of iterations, whereas accumulating synthetic data alongside the real data bounds the error by a finite ceiling independent of the iteration count, so that collapse disappears. A stronger follow-up result holds that, so long as data accumulate, the risk likely does not diverge even as human data become a vanishing fraction of the corpus.

This replace-versus-accumulate distinction is the laboratory edition of the present principle. Sever exterior contact (replace, discarding the real grounding) and the system collapses; preserve exterior contact (accumulate, never discarding the real grounding) and it stabilizes. To this one must add the final reading that neither doomers nor enthusiasts tend to state: accumulation buys a finite ceiling — “it does not get worse” — not “it gets better.” Stability needs only the preservation of old grounding; progress needs the injection of new grounding. Recycling does neither. Hence even this optimistic result confirms, rather than refutes, that a closed loop cannot self-develop.

Finally, this conclusion relocates the locus of risk. The true threat was never “AI improves itself.” First, a tool is not neutral. A sufficiently powerful tool, without becoming the selector, deforms the fitness landscape, and can deform it faster than the selection process can metabolize. The genuine hazard is generation (tooling) outrunning validation (selection) — a failure mode less dramatic than a singularity but more insidious — and what guards against it is not a limit in the AI but the speed of governance. Second, as model collapse shows, the more such tools are deployed, the more AI-generated output homogenizes the data environment, so that the tool depletes the very diversity of grounding on which it depends: the well poisons itself. The better a simulation imitates collective intelligence, the more it destroys the environment in which genuine collective intelligence could live.

The correct design principle for self-improving AI therefore converges on a single decision: who is the judge? The judge must be neither something the system can touch nor something the designer wrote down in advance, but a reality the designer did not author. Absent that, even a sandbox tournament pitting many candidates against one another is only a more expensive mirror.

References

1. Good, I. J. (1965). Speculations Concerning the First Ultraintelligent Machine. Advances in Computers, 6, 31–88.
2. Schmidhuber, J. (2006). Gödel Machines: Fully Self-Referential Optimal Universal Self-Improvers.
3. Popper, K. R. (1972). Objective Knowledge: An Evolutionary Approach. Oxford University Press.
4. Campbell, D. T. (1974). Evolutionary Epistemology. In The Philosophy of Karl Popper.
5. Hayek, F. A. (1973). Law, Legislation and Liberty, Vol. 1: Rules and Order. University of Chicago Press.
6. Taleb, N. N. (2007). The Black Swan. Random House. / (2012). Antifragile. Random House.
7. Page, S. E. (2007). The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies. Princeton University Press.
8. Cover, T. M., & Thomas, J. A. (2006). Elements of Information Theory (2nd ed.). Wiley.
9. Novikov, A., et al. (2025). AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery. Google DeepMind.
10. Anthropic / The Anthropic Institute (2026). When AI Builds Itself.
11. Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y. (2024). AI models collapse when trained on recursively generated data. Nature, 631, 755–759.
12. Gerstgrasser, M., Schaeffer, R., Dey, A., Rafailov, R., et al. (2024). Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data. arXiv:2404.01413.

CAIDE Systems, Inc. — Published July 27, 2026