Judgement and Model Minimisation
Abstract
Shannon’s information theory tells us what a channel can carry. It does not explain why particular channels exist, or who is connected to whom. For a century, computer science studied engineered communication and organisational science studied human institutions — largely apart. Agentic AI collapses that separation: multi-agent systems import org-chart templates, and organisations adopting AI become measurable communication systems.
Both are information topographies. The judgement layer is where authority over flow is exercised. The Good Regulator theorem says a good regulator must be a model of the system it regulates — so if humans remain the regulators, the systems they oversee must stay modellable. That is the case for model minimisation: not anti-capability, but the condition under which judgement remains solvent. When we automate past that condition without escalation paths, agentic debt accrues.
Opening
Information theory characterised channels as given (Shannon, 1948). Cybernetics said what communication is for (Wiener, 1948). Organisational science documented how humans adapt structure under limited attention (March and Simon, 1958; Simon, 1947). Multi-agent learning showed protocols emerging under task pressure. Each tradition held part of the picture. None provided a shared generative account of who communicates with whom, with what authority.
Engineered AI systems and human institutions can now be studied as instances of one phenomenon: communication structure under constrained information dynamics. Trent does not need the full mathematics — but it does need the implication: your security and product problems are organisational-design problems, and your organisational-design problems are now instrumentable like software.
Motivation: communication as design
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.
Melvin Conway (Conway, 1968)
The law is cited in the classic software engineering text, The Mythical Man-Month (Brooks, 1975). Because the design that occurs first is almost never the best possible, flexibility of organisation is important to effective design (Conway, 1968).
Service-oriented architecture, two-pizza teams, explicit interfaces. Now becoming a two-slice team. Amazon treated communication topology as a design variable (Bryar and Carr, 2021). That was a purposeful intervention in an information topography. AI changes the assumptions behind that memo: digital steerage is no longer limited to software engineers. So what should such a memo look like today?
From channels to topography
Shannon showed that information can be measured on a common scale, and that reliable communication is possible over noisy channels up to a capacity limit (Shannon, 1948). Upper bounds (capacity), lower bounds (communication complexity), and representation–relevance trade-offs all assume a channel. The missing piece is a generative account of structure: who communicates with whom, in what form, with what authority — as an outcome of information demand and communication cost.
A saturated channel is a bottleneck — a manager, a review board, a regulatory interface accumulating backlog. When AI lowers the cost of some channels and raises the demand on others, the topography reorganises whether you intend it or not. Drawing that map is simultaneously a CS instrumentation task and an organisational diagnosis.
The judgment layer
Judgment Layer: Organisations as emulsions
Think of an organisation as an emulsion. One phase is the automatable routines — rule-based decisions, standard workflows, repeatable processes. The other phase is the irreducible human context — tacit norms, exceptions, escalation paths, and situational awareness that rarely makes it into documentation. In a healthy organisation these two phases are held in a stable mixture. The judgment layer is the interface between them: the decisions that route a situation from routine handling to human attention.
When we automate workflows with agentic AI, we are in effect trying to distil the automatable phase and hand it to a system. But the emulsion is mixed: the judgment calls are tangled up with the routine steps. Automating without first extracting and formalising those judgment calls does not eliminate them — it just makes them invisible. That invisibility is agentic debt.
The judgment layer is not one decision but a distributed set of micro-interventions embedded in the fabric of how work flows through an organisation. Security teams accumulate this knowledge over years: which alerts are usually noise, which anomalies are actually fine because of a known upstream cause, which vendor exceptions are pre-approved. None of it is written down. It lives in the heads of experienced operators and in the patterns of who emails whom when something looks off.
For a security context, this is especially acute. The same log pattern can be routine or catastrophic depending on context that is not in the log. The judgment layer is the part of the organisation that holds that context.
Institutional tacit knowledge
This motivates an emulsion metaphor: organisations are a stable mixture of automatable routines and irreducible human context. In current institutions, the interface between these routine decisions and the judgment is mixed, like an emulsion. When we replace these decisions with orchestrated agents without understanding the judgment interventions we accumulate agentic debt. Paying down agentic debt means extracting the tacit judgment layer into explicit policies, evidence requirements, and reversible action boundaries.
Good regulators and model minimisation
Conant and Ashby: a regulator that successfully holds a system within bounds must, in effect, contain a model of that system (Conant and Ashby, 1970). An information-theoretic reading is that optimal regulation supports \(H(R|S)=0\) — the regulator’s action can be a deterministic function of system state. For Trent the practical reading is sharper. If institutional judgement is still meant to regulate agentic systems, those systems must remain modellable by the people who own the risk. Opacity is not merely an explainability preference; it is a failure of the regulator condition.
Across cybernetics and organisational science this is the same object: Beer’s live regulator (Beer, 1979, 1972); Ashby’s law of requisite variety (Ashby, 1956); uncertainty absorption and exception handling (March and Simon, 1958); Galbraith’s information-processing view of organisation design (Galbraith, 1974); and the boundary between routine cases and escalated cases in knowledge hierarchies (Garicano, 2000). Agentic AI puts new stress on it because language becomes action at machine speed.
Model minimisation is the engineering corollary of the Good Regulator theorem. It is not aesthetic minimalism and not anti-capability. Capability that cannot be modelled by the judgement layer is capability the institution cannot regulate. The unit of design is the judgement-preserving interface — including an explicit abstain / escalate path.
Agentic debt
Amazon compensated for devolved power with an extreme emphasis on judgement — leadership principles that push every employee to see themselves as owning outcomes (Bryar and Carr, 2021). Devolving authority to agents needs compensating mechanisms of the same kind: explicit escalate, contest, and halt paths. That is organisational design expressed as system policy. Agentic debt is the accrued cost when the judgement layer is displaced by automated processes that lack the authority and knowledge to contest outputs (Lawrence and Montgomery, 2026).
Agentic debt is what accrues when we violate the Good Regulator condition (Conant and Ashby, 1970) in practice: the attenuator still runs, but the institution no longer holds a live model of what it is approving (Lawrence and Montgomery, 2026). The formal org chart and the measured topography diverge.
Discussion
Thanks!
For more information on these subjects and more you might want to check the following resources.
- company: Trent AI
- book: The Atomic Human
- twitter: @lawrennd
- podcast: The Talking Machines
- newspaper: Guardian Profile Page
- blog: http://inverseprobability.com