The result that first made the self-play approach impossible to dismiss was not a surprise to everyone who saw it coming, but the magnitude was. An agent trained entirely through competition against earlier versions of itself, with no human-provided move annotations and no pre-programmed search heuristics, achieved performance across the full breadth of a game that had been played by humans for millennia. The signal for improvement came entirely from game outcomes — wins, losses, draws — not from any human's judgment about which moves were instructive.
The obvious question, which we asked ourselves when thinking about founding this laboratory in 2021, was: why should this principle be specific to games?
The obvious answer — that games have clear rules and unambiguous outcome signals, while the world does not — is correct but incomplete. It identifies a genuine constraint without exhausting the space of what can be done within and around that constraint.
The essential structure of self-play
To ask whether self-play generalises, you need to be precise about what you are trying to generalise. Self-play in game-playing agents has several components that are worth separating:
First, there is the curriculum generation mechanism. As an agent improves, it faces stronger opponents, because its opponents are copies of itself. The difficulty of the training problem tracks the capability of the learner. This is automatic and does not require human curriculum design.
Second, there is the training signal. Wins and losses constitute a reward signal that is unambiguous and dense relative to many real-world task formulations. The signal is sparse — one per game — but it is clean and does not require any labelling.
Third, there is the exploration mechanism. The adversarial structure of two-player games means that each agent is constantly probing the other for weaknesses. The exploration that results is not random; it is directed toward finding states that the opponent handles poorly. This generates rich coverage of the state space in a structured way.
None of these three components requires a game in the narrow sense. Each can be instantiated — in modified form — outside the game-playing context.
Curriculum without an opponent
In single-agent settings, the adversarial structure that generates the curriculum in two-player self-play is absent. But curriculum generation is not the same as adversarial structure. Any mechanism that generates training problems of increasing difficulty proportional to current ability is performing the curriculum function.
One direction we are exploring is what might be called task self-generation: a system that learns to propose challenges for itself based on its own current competence boundary. The idea is that a well-calibrated model of the agent's own knowledge — a metacognitive component — can identify the regions of the state space that are currently neither trivially solvable nor completely beyond reach. Problems in that intermediate zone are maximally informative for learning.
This is not a solved problem. The challenge is building a competence model that is accurate enough to be useful for curriculum generation without being so expensive that it undermines the efficiency of the overall learning process. But the direction is meaningful, and there are partial results from the broader field — work on competence-based intrinsic motivation and learning progress signals — that suggest the problem is tractable.
Signal without explicit reward
In games, the win/loss signal is clean and universally defined. Outside games, the analogous signal — what counts as progress, what counts as a good representation — is harder to define without either reintroducing human annotation or accepting a much noisier and domain-specific proxy.
There are several candidate approaches. Prediction-based objectives — learn a representation from which future states are predictable — provide signal without external annotation, though they tend to be dominated by easy-to-predict but uninformative features (colour histograms, low-frequency signals) unless carefully regularised. Contrastive objectives define positive and negative pairs based on data augmentation or known structure, then train representations that are invariant to the irrelevant differences and sensitive to the relevant ones — the hard part being defining what counts as irrelevant.
None of these fully replaces the clean signal of game outcome. But the question is not whether they match game-play in signal quality. The question is whether they provide signal rich enough to drive useful representation learning in domains where game-style outcome signals are unavailable. The empirical evidence from the past several years of self-supervised learning research suggests the answer is yes, at least in many settings.
A scenario worth considering
Consider a system learning to model the dynamics of a cellular environment — how proteins interact, how signals propagate, how perturbations affect downstream state. No labelled training set exists for most cellular states; the experimental cost of generating ground-truth measurements is high and coverage is sparse. A system learning through self-play-inspired mechanisms might work as follows: one module predicts the effect of a perturbation; an adversarial module attempts to identify perturbations where the first module's prediction is worst-calibrated. The adversarial module's output drives the selection of experiments (in simulation or, eventually, in a wet-lab feedback loop), creating a curriculum of informative questions. The prediction module improves in response.
This is a sketch, not a deployed system. But it illustrates the key move: the adversarial dynamic generates structure that guides learning in the absence of a pre-labelled dataset. The opponent is not a human annotator and not a game engine. It is a component of the learning system itself.
Where the analogy breaks — and why that matters
We do not want to overextend the self-play analogy. Games have properties that are genuinely unusual: they are fully observable (in the cases where self-play has worked most dramatically), their state spaces are large but finite and well-defined, and their outcome signals are perfectly consistent. Real-world domains are often partially observable, have state spaces that are not just large but ill-defined, and have outcome signals that are noisy, delayed, or proxy measures for something else.
The challenge for the generalisation project is to preserve the useful structure of self-play — the automatic curriculum, the exploration pressure, the signal that does not require human annotation — while relaxing the assumptions that make those properties easy to implement in the game-playing setting. This is harder than it sounds. Each relaxation introduces new problems, and the compound difficulty of all the relaxations simultaneously is substantial.
We think the project is worth pursuing precisely because the payoff is large. A system that can acquire knowledge of complex domains through self-generated experience, without requiring either labelled data or a cleanly specified reward function, would operate in a qualitatively different regime from systems that rely on either. It would be capable of improvement in domains where human understanding is itself still forming — where the right labels have not yet been decided and where the right reward function has not yet been formulated.
That is the regime where the most interesting problems live. Getting there from self-play in games is a long way. But the path starts from a principle that has already been demonstrated to work in at least one important case, and that is a better starting point than most.
