AI and Security: From Bandwidth to Practical Implications

Agentic AI, HAMs, and the new attack surface

Neil D. Lawrence

Agentic AI Summit 2026, UC Berkeley

Frame: why agentic AI shifts security

  • Modern AI systems operate at machine bandwidth; humans interpret at human bandwidth.
  • Incidents unfold faster than review cycles and slower-than-real-time approvals.
  • Agentic systems turn text into actions (tools, APIs, workflows), expanding the attack surface.

Information Theory and AI

  • Claude Shannon developed information theory at Bell Labs
  • Information measured in bits, separated from context
  • Makes information fungible and comparable

Information Transfer Rates

  • Humans speaking: ~2,000 bits per minute
  • Machines communicating: ~600 billion bits per minute
  • Machines share information 300 million times faster than humans

New Flow of Information

New Flow of Information

Evolved Relationship

Evolved Relationship

Bandwidth vs Complexity
bits/min \(100 \times 10^{-9}\) \(2,000\) \(600 \times 10^9\)

A lens: Human Analogue Machines (HAMs)

  • HAMs amplify capability: summarise, plan, draft, search, coordinate.
  • HAMs amplify vulnerability: persuasion, authority bias, social engineering.
  • Security becomes interface security: what the system can be induced to do, and who can induce it.

Human Analogue Machine

Human Analogue Machine

  • A human-analogue machine is a machine that has created a feature space that is analagous to the “feature space” our brain uses to reason.

  • The latest generation of LLMs are exhibiting this charateristic, giving them ability to converse.

Heider and Simmel (1944)

Counterfeit People

  • Perils of this include counterfeit people.
  • Daniel Dennett has described the challenges these bring in an article in The Atlantic.

Psychological Representation of the Machine

  • But if correctly done, the machine can be appropriately “psychologically represented”

  • This might allow us to deal with the challenge of intellectual debt where we create machines we cannot explain.

HAM

HAM

Three phases of security change

  • Use GenAI to compress bandwidth for defenders: triage, summarise, correlate, explain.
  • Turn logs and alerts into decision-ready narratives with provenance.
  • Main risk: over-trust and automation bias (false confidence at scale).
  • Secure summarisation: bounded context, redaction, and provenance links to primary logs.
  • Analyst copilots: draft investigations, but keep approvals and irreversible actions human.
  • “Faster-than-human” response: pre-authorise containment actions, not remediation.
  • Prompt injection becomes an operational threat when the model has tools.
  • Indirect prompt injection via documents/web pages contaminates the instruction stream.
  • Data exfiltration shifts from perimeter breach to model-mediated leakage.

Computer Science Paradigm Shift

  • Von Neuman Architecture:
    • Code and data integrated in memory
  • Today (Harvard Architecture):
    • Code and data separated for security

Computer Science Paradigm Shift

  • Machine learning:
    • Software is data
  • Machine learning is a high level breach of the code/data separation.
  • Treat prompts, tool outputs, and retrieved documents as untrusted inputs.
  • Make instruction hierarchy explicit: system/developer/user/tool/data.
  • Apply least privilege to tools; require confirmations for high-impact actions.
  • Re-design systems for delegation with accountability.
  • Make authority boundaries explicit: who can cause which actions, with which evidence.
  • Build for recovery: audit trails, reversible actions, and containment-by-default.

Intellectual Debt

Technical Debt

  • Compare with technical debt.
  • Highlighted by Sculley et al. (2015).

Separation of Concerns

  • Decompose your complex problem/task into parts.
  • Each part manageable (e.g. by a small team)
  • Recompose to solve total problem

Addresses Complex Challenge

  • Highly successful approach to complex tasks.
  • Tuned to the human bandwidth limitation.
  • But the whole system still hard to understand.

Intellectual Debt

  • Technical debt is the inability to maintain your complex software system.
  • Intellectual debt is the inability to explain your software system.
  • Agentic AI can pay down technical and intellectual debt: compressing bandwidth, making work legible, and reducing coordination overhead.
  • But it can create agentic debt: delegation without clear authority boundaries, least-privilege tooling, and reversible actions.
  • Pay it down with explicit instruction hierarchy, audited tool calls, and rollback-by-default workflows.

Lancelot

  • Separate “thinking” from “acting”: plan, justify, then execute with logged evidence.
  • Design for rollback: reversible actions and short-lived credentials.
  • Make audits cheap: every action produces an explanation and a trace.

Case studies and practical takeaways

  • What happened can matter less than how quickly it unfolded.
  • At scale, the defender’s bottleneck is often interpretation and coordination, not detection.
  • Modern attackers exploit organisational latency (handoffs, approvals, ambiguity).
  • Agentic workflows chain: retrieval → reasoning → tool use → action.
  • The critical security question: who can influence what the agent believes and what it does?
  • Threat model: indirect prompt injection, authority confusion, and data boundary violations.
  • Bandwidth mismatch is the core risk: systems move faster than human sense-making.
  • Agentic AI turns text attacks into action attacks: model + tools = new threat model.
  • Design for legibility: instruction hierarchies, provenance, and auditable action boundaries.
  • Prefer reversible, least-privilege delegation with strong defaults and fast containment.

Thanks!

  • company: Trent AI
  • book: The Atomic Human
  • twitter: @lawrennd
  • The Atomic Human pages topography, information 34-9, 43-8, 57, 62, 104, 115-16, 127, 140, 192, 196, 199, 291, 334, 354-5 , anthropomorphization (‘anthrox’) 30-31, 90-91, 93-4, 100, 132, 148, 153, 163, 216-17, 239, 276, 326, 342, Human evolution rates 98-99, Psychological representation of Ecologies 323-327, ignorance: HAMs 347, test pilot 163-8, 189, 190, 192-3, 196, 197, 200, 211, 245, psychological representation 326–329, 344–345, 353, 361, 367, human-analogue machine 343–5, 346–7, 358–9, 365–8, human-analogue machine (HAMs) 343-347, 359-359, 365-368, intellectual debt 84, 85, 349, 365, separation of concerns 84-85, 103, 109, 199, 284, 371, intellectual debt 84-85, 349, 365, 376.
  • newspaper: Guardian Profile Page
  • blog: http://inverseprobability.com

References

Heider, F., Simmel, M., 1944. An experimental study of apparent behavior. The American Journal of Psychology 57, 243–259. https://doi.org/10.2307/1416950
Scally, A., 2016. Mutation rates and the evolution of germline structure. Philosophical Transactions of the Royal Society B 371. https://doi.org/10.1098/rstb.2015.0137
Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.-F., Dennison, D., 2015. Hidden technical debt in machine learning systems, in: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (Eds.), Advances in Neural Information Processing Systems 28. Curran Associates, Inc., pp. 2503–2511.