The Inaccessible Game

• Avoid external structure.
• Represent information loss (via Entropy)
• Enforce information conservation (information isolation, Lawrence (2025))

❮ ❯

• Correlation is when two variables are dependent

❮ ❯

• Dice example: Average result 4.5 instead of 3.5
• Constraints:
  • \(\sum_{n=1}^6 P_n = 1\) (normalization)
  • \(\sum_{n=1}^6 nP_n = 4.5\) (observed average)

Forbidden:

• External observer
• Pre-specified outcome space
• Privileged decomposition
• External time parameter

No appeal to structure outside the game

• We don’t see see the outcome space
• But we summarise it using entropy
• Entropy gives a single number measuring uncertainty

• Each distribution gets a score \(\rightarrow\) compare them
• Induces ordering from low to high uncertainty
• Formally: a functor from FinProb to \((\Re, \leq)\)

• Marginal entropy of variable \(i\): \(h_i\)
• Joint entropy of system: \(H\)
• Multiinformation \(I = \sum_i h_i - H\)

• Two characterisations of information loss:
  • Baez et al. (2011): \(\textsf{FinProb}\), Shannon entropy
  • Parzygnat (2022): \(\textsf{NCFinProb}\), von Neumann entropy
• Which is the right internal language for the game?

• Three axioms \(\rightarrow\) \(F(f) = c(H(p) - H(q))\) uniquely
• \(\textsf{FinProb}\) is cartesian
• Canonical copying: \(\Delta_A : A \to A \times A\) available for free

• Analogous axioms \(\rightarrow\) \(F(f) = c(S(\rho) - S(\sigma))\) via von Neumann entropy
• \(\textsf{NCFinProb}\) is symmetric monoidal
• No canonical copying: \(A \to A \otimes A\) not available by default
• This is the quantum no-cloning property

• Lawvere diagonalisation needs: self-application \(\equiv\) canonical copying
• \(\textsf{FinProb}\) (cartesian): copying available \(\to\) paradox constructible \(\to\) external adjudication required
• \(\textsf{NCFinProb}\) (symmetric monoidal): no canonical copying \(\to\) diagonalisation blocked \(\to\) no-barber satisfied

• Cannot inspect the configuration, \(\rho\), directly
• But entropy requires no external reference
• Internally accessible: no external adjudicator needed

• \(S\) maps each configuration to \([0, \infty)\)
• Pull back the order on \(\mathbb{R}\): \(\rho \leq \rho'\) iff \(S(\rho) \leq S(\rho')\)
• Reflexive ✓, Transitive ✓, Antisymmetric ✗ (distinct \(\rho\) can share entropy)
• Preorder (not a partial order): configurations form a proset

• Isoentropy: \(\rho \sim \rho'\) iff \(S(\rho) = S(\rho')\) — an equivalence class
• Quotient \([\rho]\): each class is one point \(\to\) antisymmetry holds \(\to\) poset
• Since \(S\) maps into \(\mathbb{R}\): every two classes are comparable \(\to\) chain (total order)
• From outside: only the chain of entropy levels is visible

• Bottom (\(S=0\)): pure state — fully ordered, single configuration
• Higher rungs: mixed states — more equivalence classes at each level
• No canonical top: maximum entropy grows with system dimension
• Dynamics = movement along the ladder
• Functor: \(S : \textsf{NCFinProb} \to (\mathbb{R}_{\geq 0}, \leq)\)

\[ \sum_{i=1}^N h_i = C \]

What does this conservation imply for dynamics?

\[ I = \sum_{i=1}^N h_i - H \]

• \(I = 0\): Independent variables
• \(I > 0\): Correlated variables
• Larger \(I\) = more correlation

Measures “shared information”

\[ I + H = C \]

Conserved quantity splits into two parts

Analogy to classical mechanics:

• Energy: \(T + V = E\)
• Information: \(I + H = C\)

• Second law: Entropy increases (\(\dot{H} > 0\))
• Conservation: \(I + H = C\) (constant)
• Therefore: Correlation decreases (\(\dot{I} < 0\))

• Compressed spring \(\rightarrow\) released energy
• Correlated variables \(\rightarrow\) independent variables
• Potential \(\rightarrow\) kinetic

• Can derive GENERIC-like dynamics.
• Origin suggests von Neumann entropy more natural than Shannon.

• Nice connections between.
  • Thermodynamics and Inference (Jaynes).
  • Information Geometry and GENERIC.
  • Inaccessibility and noncommutative probability.

• In certain thermodynamic limits:
  • Marginal entropy conservation \(\equiv\) Energy conservation

See Lawrence (2025)

• 1925: Promises of perpetual motion cars
• 2025: Promises of superintelligence singularity
• Same fundamental impossibility?

\[\frac{\text{d}H}{\text{d}t} \geq 0\]

• Entropy always increases
• No motion without entropy production
• No work without energy input

Maxwell’s Demon:

• “Intelligent” entity that violates 2nd law
• Resolution: Landauer’s principle?
• Information erasure requires energy

• Intelligence has thermodynamic cost
• Information processing has physical limits

• Thermodynamics limits mechanical engines

• Information theory limits information engines

Same kind of fundamental constraint

• Acquire information (sensing)
• Store information (memory)
• Process information (computation)
  
• Erase information (memory mgmt)
• Act on information (output)

What is the thermodynamic cost?

Erasing 1 bit requires: \(Q \geq k_BT\log 2\)

• Not engineering limitation
• Fundamental thermodynamic bound
• Entropy must go somewhere

At room temperature: \(\sim 3 \times 10^{-21}\) Joules/bit