Abstract

In 1925 the automobile was already transforming the world. Imagine if promises had been made of a car that needed no fuel. We would call that a perpetual motion machine, and we know why it is impossible: the second law of thermodynamics. A century later the analogous promise is superintelligence. I have no doubt that AI will transform our world as much as the automobile did. I also have no doubt that the promise of unbounded intelligence is as empty as the promise of unbounded motion.

Thermodynamics limits mechanical engines. Information theory should limit information engines. To make that claim precise we need a setting in which the rules cannot appeal to an external referee — a self-governing system, in the same sense that the laws of physics cannot pause to check VAR. That is the inaccessible game: a zero-player, information-theoretic dynamical system whose only admissible rules are those that can be adjudicated from within.

From a tautology — self-governing systems cannot refer to external arbitration — a surprisingly specific structure follows. A “no-barber” principle, in the spirit of Russell’s paradox, forbids external observers, pre-specified outcome spaces, and privileged clocks. What remains is information isolation, a conservation law on marginal entropy, and, in the thermodynamic limit, an equivalence between that conservation and energy. The same constraints that rule out perpetual motion begin to look like constraints on intelligence.

Motivation

Perpetual Motion and Superintelligence

Imagine in 1925 a world where the automobile is already transforming society, but big promises are being made for things to come. The stock market is soaring, the 1918 pandemic is forgotten. And every major automobile manufacturer is investing heavily on the promise they will each be the first to produce a car that needs no fuel. A perpetual motion machine.

Well, of course that didn’t happen. But I sometimes wonder if what we’re seeing today 100 years later is the modern equivalent of that. In 2025 billions are being invested in promises of superintelligence and artificial general intelligence that will transform everything.

We know why perpetual motion is impossible: the second law of thermodynamics tells us that entropy always increases. So we can’t have motion without entropy production. No matter how clever the design, you cannot extract energy from nothing, and you cannot create a closed system that does useful work indefinitely without an external energy source.

How might we make an equivalent statement for the bizarre claims around superintelligence? Some inspiration comes from Maxwell’s demon, an “intelligent” entity which operates against the laws of thermodynamics. The inspiration comes because the demon suggests that for the second law to hold there must be a relationship between the demon’s decisions and thermodynamic entropy.

One of the resolutions comes from Landauer’s principle, the notion that erasure of information requires heat dissipation. This suggests there are fundamental information-theoretic constraints on intelligent systems, just as there are thermodynamic constraints on engines.

I’ve no doubt that AI technologies will transform our world just as much as the automobile has. But I also have no doubt that the promise of superintelligence is just as silly as the promise of perpetual motion. The inaccessible game provides one way of understanding why.

Information-Theoretic Limits on Intelligence

Just as the second law of thermodynamics places fundamental limits on mechanical engines, no matter how cleverly designed, the idea is that information theory places fundamental limits on information engines, no matter how cleverly implemented.

What Intelligent Systems Must Do

Any intelligent system, whether biological or artificial, must perform certain fundamental operations:

1. Acquire information from its environment (sensing, observation)
2. Store information about the world (memory)
3. Process information to make decisions (computation)
4. Erase information to make room for new data (memory management)
5. Act on the world using the processed information

Each of these operations has information-theoretic costs that cannot be eliminated by clever engineering.

Landauer’s Principle

Landauer’s principle (Landauer, 1961) establishes that erasing one bit of information requires dissipating at least \(k_BT\log 2\) of energy as heat, where \(k_B\) is Boltzmann’s constant and \(T\) is temperature.

This isn’t an engineering limitation, it’s a fundamental consequence of the second law. To reset a bit to a standard state (say, always 0) requires reducing its entropy from 1 bit to 0 bits. That entropy must go somewhere, and it ends up as heat in the environment.

This doesn’t mean AI can’t be powerful or transformative — internal combustion engines transformed the world despite thermodynamic limits. But it does mean there are hard bounds on what’s possible, and claims that ignore these bounds are as unrealistic as promises of perpetual motion.

Information

Thermalisation from Different Initial Conditions

This simulation places exactly nine billiard balls on a 3×3 grid, each coloured according to its position. The 3×3 histogram grid tracks, for each ball, the cumulative 2-D velocity distribution \((v_x, v_y)\) it has visited since the last reset.

The entropy \(H(v_x, v_y)\) for each ball is shown in the top-left of its panel. At the start, when all balls move identically, every panel shows a single bright dot near zero entropy. As elastic collisions redistribute energy the dots spread outward, tracing the Maxwell–Boltzmann circle, and entropy climbs toward its maximum.

The coloured dot in the top-right corner of each panel matches the ball’s colour on the main canvas, making it easy to follow individual balls.

Use the Display dropdown to switch between the 2D joint distribution \(p(v_x, v_y)\) (heatmap) and the two 1D marginals \(p(v_x)\) and \(p(v_y)\) overlaid as bar charts. Both marginals are expected to converge to the same symmetric distribution; the coloured bars show \(p(v_x)\) (ball colour) and the dark outline shows \(p(v_y)\). The entropy labels \(H_x\) and \(H_y\) confirm that the two components thermalise at the same rate.

Use the Initialisation dropdown to choose how the balls start:

For the four directional cases all nine histograms start at the same point, yet rapidly diverge and then converge to the same circular distribution. For the propellor cases adjacent balls start with very different velocity directions — the corner and edge balls even start at nearly opposite velocities — and yet all nine panels converge to the same equilibrium blob.

Notice that this system is ergodic: in the long-run distribution of each ball’s velocity is independent of the initial conditions and identical for all balls, even though the path to equilibrium differs.

Reset Pause / Resume Skip 1000s

Initialisation: From top ↓ From bottom ↑ From left → From right ← Clockwise ↻ Counter-CW ↺ Display: 1D marginals p(vₓ) & p(vᵧ) 2D joint p(vₓ,vᵧ)

Figure: Nine billiard balls on a 3×3 grid. The histogram grid tracks each ball’s cumulative \((v_x, v_y)\) velocity distribution. Entropy per ball rises from near zero (single bright dot at the initial velocity) to the Maxwell–Boltzmann value as collisions thermalise the gas. Use the Initialisation dropdown to compare directional starts (all balls move the same way) with propellor starts (adjacent balls move in opposite directions): all initial conditions converge to the same equilibrium, demonstrating ergodicity.

Jaynes and Maximum Entropy

Figure: Ed Jaynes who developed the maximum entropy principle

Maximum Entropy Motivation

Ed Jaynes (Jaynes, 1957), proposed a foundation for statistical mechanics based on information theory. Jaynes recast that the problem of assigning probabilities in statistical mechanics as a problem of inference with incomplete information.

A central problem in statistical mechanics is assigning initial probabilities when our knowledge is incomplete. The canonical example is if we know only the average energy of a system, what probability distribution should we use? Jaynes argued that we should use the distribution that maximises entropy subject to the constraints of our knowledge.

Jaynes illustrated the approach with a simple example. If a die has been tossed many times, with an average result of 4.5 rather than the expected 3.5 for a fair die. What probability assignment \(P_n\) (\(n=1,2,...,6\)) should we make for the next toss?

We need to satisfy two constraints \[\begin{align} \sum_{n=1}^6 P_n &= 1 \\ \sum_{n=1}^6 n P_n &= 4.5 \end{align}\]

Many distributions could satisfy these constraints, but which one makes the fewest unwarranted assumptions? Jaynes argued that we should choose the distribution that is maximally noncommittal with respect to missing information - the one that maximises the entropy, \[\begin{align} S_I = -\sum_{i} p_i \log p_i \end{align}\] This principle leads to the exponential family of distributions, which in statistical mechanics gives us the canonical ensemble and other familiar distributions.

Die Roll Simulation

This simulation illustrates the maximum entropy principle through Jaynes’ dice example (Jaynes, 1957). A fair die has expected outcome 3.5; the Jaynes example asks: if we know only that the average outcome is 4.5, what probability distribution \(P_n\) over the six faces should we assign?

The answer is the maximum-entropy distribution subject to the constraint \(\sum_{n=1}^6 n P_n = 4.5\), which belongs to the exponential family: \[\begin{align} P_n = \frac{e^{\lambda n}}{Z(\lambda)}, \qquad Z(\lambda) = \sum_{n=1}^6 e^{\lambda n} \end{align}\] where \(\lambda > 0\) is chosen so the mean constraint is satisfied. This avoids any unwarranted assumption beyond the available data.

click die or button to roll

Roll Roll ×100

Rolls: 0
Sample mean: —
H(p): —

Reset history

---

Uniform (mean 3.5) Jaynes: mean 4.5 Simple 50/50: mean 4.5 Low-biased: mean 2

Outcome weights (auto-normalised to probabilities)

Figure: Interactive die-roll simulation. Click the die or press Roll to sample from the configured distribution. The histogram shows empirical relative frequencies (coloured bars) overlaid on the theoretical probabilities (dashed outlines). Use the sliders to set arbitrary outcome weights, or click a preset to load the uniform distribution (mean 3.5), the Jaynes maximum-entropy distribution (mean 4.5), the simple 50/50 distribution (faces 4 and 5 equally, mean 4.5 with minimal entropy), or a low-biased distribution (mean 2).

The General Maximum-Entropy Formalism

For a more general case, suppose a quantity \(x\) can take values \((x_1, x_2, \ldots, x_n)\) and we know the average values of several functions \(f_k(x)\). The problem is to find the probability assignment \(p_i = p(x_i)\) that satisfies \[\begin{align} \sum_{i=1}^n p_i &= 1 \\ \sum_{i=1}^n p_i f_k(x_i) &= \langle f_k(x) \rangle = F_k \quad k=1,2,\ldots,m \end{align}\] and maximises the entropy \(S_I = -\sum_{i=1}^n p_i \log p_i\).

Using Lagrange multipliers, the solution is the generalised canonical distribution, \[\begin{align} p_i = \frac{\exp(-\lambda_1 f_1(x_i) - \ldots - \lambda_m f_m(x_i))}{Z(\lambda_1,\ldots,\lambda_m)} \end{align}\] where \(Z(\lambda_1,\ldots,\lambda_m)\) is the partition function, \[\begin{align} Z(\lambda_1,\ldots,\lambda_m) = \sum_{i=1}^n \exp(-\lambda_1 f_1(x_i) - \ldots - \lambda_m f_m(x_i)) \end{align}\] The Lagrange multipliers \(\lambda_k\) are determined by the constraints, \[\begin{align} \langle f_k \rangle = -\frac{\partial}{\partial \lambda_k}\log Z(\lambda_1,\ldots,\lambda_m) \quad k=1,2,\ldots,m. \end{align}\] The maximum attainable entropy is \[\begin{align} (S_I)_{max} = \log Z + \sum_{k=1}^m \lambda_k \langle f_k \rangle. \end{align}\]

\[ p_i = \frac{\exp(-\lambda_1 f_1(x_i) - \ldots - \lambda_m f_m(x_i))}{Z(\lambda_1,\ldots,\lambda_m)} \] \[ Z(\ldots) = \sum_{i=1}^n \exp(-\lambda_1 f_1(x_i) - \ldots - \lambda_m f_m(x_i)) \] \[ \langle f_k \rangle = -\frac{\partial}{\partial \lambda_k}\log Z(\lambda_1,\ldots,\lambda_m) \quad k=1,2,\ldots,m. \]

Figure: Samples from independent Gaussian variables that represent horizontal and vertical velocities when our system is at equilibrium.

Sampling Two Dimensional Variables

Figure: Samples from correlated Gaussian variables that represent vertical and horizontal velocity.

Figure: Two independent Gaussians for the \(x\) and \(y\) velocity of a ball.

Figure: A correlated Gaussian for the \(x\) and \(y\) velocity of a ball. If all balls were correlated in this way, this would imply that the whole box is moving towards the upper right or bottom left.

Figure: An anti-correlated Gaussian for the \(x\) and \(y\) velocity of a ball. If all balls were anti-correlated in this way, this would imply that the whole box is moving towards the upper left or bottom right.

The Classical Observer

Figure: Here the observer is monitoring the movements of the particles. We’ve plotted the velocities alongside the 1 standard deviation contour of their theoretical distribution.

The Classical Observer - Correlated

Figure: Again the observer is monitoring the movements of the particles, but here their motion is correlated (\(\rho=0.95\)).

The Classical Observer - Anti-correlated

Figure: Here the observer is monitoring the movements of the particles, but here their motion is anti-correlated (\(\rho=-0.95\)).

The Inaccessible Game

The Munchkin Provision

Without such consistency, we would require what we might call a “Munchkin provision.” In the Munchkin card game (Jackson, 2001), it is acknowledged that the cards and rules may be inconsistent. Their resolution?

Any other disputes should be settled by loud arguments, with the owner of the game having the last word.

Munckin Rules (Jackson, 2001)

While this works for card games, it’s unsatisfying for foundational mathematics. We want our game to be internally consistent, not requiring an external referee to resolve paradoxes.

Figure: The Munchkin card came has both cards and rules. The game explicitly acknowledges that this can lead to inconsistencies which should be resolved by the game owner.

A Tautology

Self-governing systems cannot refer to external arbitration.

While this is a tautology, we will try to formalise it through information theory. The key question is: what mathematical structure is forced on a system that cannot appeal to external adjudication?

The No-Barber Principle

In 1901 Bertrand Russell introduced a paradox: if a barber shaves everyone in the village who does not shave themselves, does the barber shave themselves? The paradox arises when a definition quantifies over a totality that includes the defining rule itself.

We propose a similar constraint for the inaccessible game: the foundational rules must not refer to anything outside themselves for adjudication or reference. Or in other words there can be no external structure. We call this the “no-barber principle” (Lawrence, 2026).

The no-barber principle says that admissible rules must be internally adjudicable: they depend only on quantities definable from within the system’s internal language, without requiring e.g. an external observer to define the co-ordinates or a privileged decomposition.

Baez-Fritz-Leinster Characterization of Information Loss

Before introducing our fourth axiom, we need to understand how information loss is measured. Baez et al. (2011) showed that entropy emerges naturally from category theory as a way of measuring information loss in measure-preserving functions. They derived Shannon entropy from three axioms, without invoking probability directly.

The Three Axioms

Let \(F(f)\) denote the information lost by a process \(f\) that transforms one probability distribution to another. The three axioms constrain the functional form of \(F\).

Axiom 1: Functoriality suggests that given a process consisting of two stages, the amount of information lost in the whole process is the sum of the amounts lost at each stage: \[ F(f \circ g) = F(f) + F(g), \] where \(\circ\) represents composition.

Axiom 2: Convex Linearity suggests that if we flip a probability-\(\lambda\) coin to decide whether to do one process or another, the information lost is \(\lambda\) times the information lost by the first process plus \((1-\lambda)\) times the information lost by the second: \[ F(\lambda f \oplus (1-\lambda)g) = \lambda F(f) + (1-\lambda)F(g). \]

Axiom 3: Continuity suggests that if we change a process slightly, the information lost changes only slightly, i.e. \(F(f)\) is a continuous function of \(f\).

The Main Result

The main result of Baez et al. (2011) is that these three axioms uniquely determine the form of information loss. There exists a constant \(c\geq 0\) such that for any \(f: p \rightarrow q\): \[ F(f) = c(H(p) -H(q)) \] where \(F(f)\) is the information loss in process \(f: p\rightarrow q\) and \(H(\cdot)\) is the Shannon entropy measured before and after the process is applied to the system.

This provides a foundational justification for using entropy as our measure of information. It is not just a convenient choice — it is the unique measure satisfying these natural requirements for measuring information loss. This is a theorem (Baez et al., 2011). The quantum analogue — replacing finite probability spaces and Shannon entropy with finite-dimensional noncommutative probability spaces and von Neumann entropy — is established by Parzygnat (2022).

The Inaccessible Game Setup

Inspired by the no-barber principle, we set up the game in a way that attempts to avoid “external structure.” The first two things we need to do this are

1. A representation of information loss
2. A prohibition of information exchange with the game

How do we obtain a representation of information loss without including external structure? We use the axiomatic frameworks of Baez et al (Baez et al. (2011)) and Parzygnat (Parzygnat (2022)). They characterise entropy through category theory frameworks that depend on three axioms. Slight differences in the axioms result in different conclusions. Baez et al conclude that difference in Shannon entropy before and after a process is applied characterises information loss. Parzygnat is inspired by Baez et al but reformulates around a different categorical object which implies von Neumann entropy.

In the game (Lawrence (2025)) we introduce information conservation based on these measures of information loss.

Information Isolation

The first three axioms of the inaccessible game, due to Baez et al. (2011), characterise information loss and justify the use of entropy. For the game itself we introduce a fourth axiom: information isolation. Just as an isolated chamber conserves mass and energy, our game is isolated from external observation. No observer outside the system can extract or inject information.

Under additional requirements of exchangeability and extensivity, information isolation implies that the total marginal entropy is conserved. For any finite sub-group of \(N\) variables the sum of marginal entropies \(\{h_i\}_{i=1}^{N}\) sums to a constant \(C\), \[ \sum_{i=1}^N h_i = C. \] The conservation law is imposed in an exchangeable form across the marginal entropies, so that it applies consistently to any finite partition drawn from a potentially countably infinite collection of variables.

The specific form \(\sum_i h_i = C\) is not an arbitrary choice. Any exchangeable quantity depending only on marginal entropies must take the form \(Q = \sum_i f(h_i)\) with the same function \(f\) for each variable. Extensivity (adding one variable increases \(Q\) by a fixed amount) forces \(f(h) = c \cdot h + \text{const}\). Requiring the law to apply consistently as the subset size varies eliminates the constant term. Setting \(c=1\) gives the unique form \(\sum_i h_i = \text{const}\). The fourth axiom is therefore the unique exchangeable, extensive, information-theoretic conservation law for an isolated system.

Information isolation can be seen as stronger than frame invariance. It eliminates not only preferred reference frames but appeal to external reference structures. All physically meaningful quantities must be internal to the system and relational rather than absolute. The variable partition \(\{X_i\}\) that enters the conservation constraint is a structural choice. It is part of the model specification, analogous to choosing a Hilbert space factorisation in quantum mechanics—rather than an externally privileged decomposition.

In traditional thermodynamics, energy conservation defines a built-in potential. Here, marginal entropy conservation plays the analogous role: it defines an intrinsic potential within the information geometry. The curvature of this potential, encoded in the Fisher information, acts as the metric governing how the system redistributes its informational content.

The Classical Observer - Inaccessible

Figure: Here the observer is blocked from monitoring anything inside the sytem.

When we don’t know what’s going on inside, we can’t express outcomes in the way we could with an observer. But we can still express entropies. This highlights an interesting characteristic of entropies. If we don’t express the probability directly, but just work with the entropies themselves, it feels like we can assess the bounds of possibility without directly expressing what’s going on.

Joint Entropy

While we don’t see the underlying probability, we can capture a class of different distirbutions by considering the mapping to the system’s joint entropy.

Think of joint entropy as a scoring system: every configuration gets a number measuring its uncertainty. Once you have that, you can line them up from least to most disordered.

The \(I + H = C\) Structure

We have established four axioms, with the fourth axiom stating that the sum of marginal entropies is conserved, \[ \sum_{i=1}^N h_i = C. \] This conservation law is the heart of The Inaccessible Game, but to understand its dynamical implications, we need to rewrite it in a more revealing form.

Multi-Information: Measuring Correlation

The multi-information (or total correlation), introduced by Watanabe (1960), measures how much the variables in a system are correlated. It is defined as, \[ I = \sum_{i=1}^N h_i - H, \] where \(H\) is the joint entropy of the full system: \[ H = -\sum_{\mathbf{x}} p(\mathbf{x}) \log p(\mathbf{x}). \]

The multi-information has a nice interpretation:

• \(I = 0\): The variables are completely independent. The joint entropy equals the sum of marginal entropies.
• \(I > 0\): The variables are correlated. Some information is “shared” between variables, so the joint entropy is less than the sum of marginals.
• \(I\) is maximal: The variables are maximally correlated (in the extreme case, deterministically related).

Multi-information is always non-negative (\(I \geq 0\)) and measures how much knowing one variable tells you about others.

Using the definition of multi-information, we can rewrite our conservation law. From \(I = \sum_{i=1}^N h_i - H\), we have: \[ \sum_{i=1}^N h_i = I + H. \] Therefore, the fourth axiom \(\sum_{i=1}^N h_i = C\) becomes: \[ I + H = C. \]

This is an information action principle. It says that multi-information plus joint entropy is conserved. This equation sits behind the dynamics of the Inaccessible Game.

This equation has the structure of an action principle in classical mechanics. In physics, total energy is conserved and splits into two parts, \[ V + T = E, \] where \(V\) is potential energy and \(T\) is kinetic energy.

The analogy for The Inaccessible Game is.

• Multi-information \(I\) plays the role of potential energy. It represents “stored” correlation structure. High \(I\) means variables are tightly coupled, like a compressed spring.
• Joint entropy \(H\) plays the role of kinetic energy. It represents “dispersed” or “free” information. High \(H\) means the probability distribution is spread out, with maximal uncertainty.

Just as a classical system evolves from high potential energy to high kinetic energy (a ball rolling down a hill), the idea in the Inaccessible Game will be that the information system evolves from high correlation (high \(I\)) to high entropy (high \(H\)).

Energy

Energy Constraints

The Conservation Law

Intelligence

The perpetual motion analogy is no longer only an analogy. Once the game is internally adjudicable, the same kind of constraint that forbids a fuel-less engine begins to look like a constraint on unbounded intelligence. Thermodynamics limits mechanical engines. Information isolation limits information engines. Superintelligence, on this view, is the claim that those limits can be wished away.

Thanks!

For more information on these subjects and more you might want to check the following resources.

• company: Trent AI
• book: The Atomic Human
• twitter: @lawrennd
• podcast: The Talking Machines
• newspaper: Guardian Profile Page
• blog: http://inverseprobability.com

References