at Amazon Thursday Thoughts on Mar 29, 2018 [reveal]
Neil D. Lawrence, Amazon and University of Sheffield

#### Abstract

What is the nature of machine intelligence and how does it differ from humans? In this talk we explore some of the differences between natural and machine intelligence.

How are we making computers do the things we used to associated only with humans? Have we made a breakthrough in understanding human intelligence?

While recent achievements might give the sense that the answer is yes, the short answer is that we are nowhere near. All we’ve achieved for the moment is a breakthrough in emulating intelligence.

## What is Intelligence? 

One challenge with the word intelligence is it means many different things to different people. My own definition of intelligence is the use of information to achieve goals more efficiently. Where information is measured in Shannon terms, and efficiency is measured in terms of use of resource, typically available energy or maybe time. The definition appeals to me because it brings about a connection between the major historic revolutions, which have been about energy, and the current changes which focus more on information. However, the definition does not specify the goal. Some prefer to try and incorporate the goal in their definition of intelligence, and that goals should be somehow emergent from intelligence. For complex tasks sub-goals are certainly emergent, but I don’t seek to incorporate a global goal in my own definition.

So where are we in terms of intelligence? There is certainly an information revolution going on, and it is causing disruption across many industries. But what we are saying about machine learning and artificial inteligence has been said before in relation to the invention of sillicon chips. I gave a version of this talk once at the BBC, and Bill Thompson was kind enough to share with me the archive of “Silicon Factor”, a series of three programs broadcast in 1980 exploring the “microelectronics revolution”. There is a remarkable similarity to everything the experts say in these programs and what experts say about AI today. And of course, these experts were right, jobs have changed, industry is different and we are still deep in the micro-electronics revolution. However, these changes do not happen overnight, those broadcasts were made nearly forty years ago. The “Fourth Industrial Revolution” is merely a continuation of an ongoing revolution in information, one which was triggered by the Silicon Factor, continued by the internet and has been given further momentum by mobile communications.

With a word like intelligence, we can’t just think about our own definition of intelligence, but we should also take into account public understanding of the world. For many people intelligence is something specific to humans, and as a result what the term refers to evolves. Artificial intelligence is a very emotive term, because it feels close to us, it makes us think that computers are doing things like us. As a result it is also a shifting definition, it comes to me “intelligence is the stuff I can do that computers can’t”. This lends a narcissistic element to our fascination with artificial intelligence, because it is also a fascination with ourselves.

A hundred years ago computers were human beings, often female, who conducted repetitive mathematical tasks for the creation of mathematical tables such as logarithms. Our modern digital computers were originally called automatic computers to reflect the fact that the intelligence of these human operators had been automated. But despite the efficiency with which they perform these tasks, very few think of their mobile phones or computers as intelligent.

Norbert Wiener launched last century’s first wave of interest in emulation of intelligence with his book “Cybernetics”. The great modern success that stemmed from that work is the modern engineering discipline of Automatic Control. The technology that allows fighter jets to fly. These ideas came out of the Second World War, when researchers explored the use of radar (automated sensing) and automatic computation for decryption of military codes (automated decision making). Post war a body of researchers, including Alan Turing, were seeing the potential for electronic emulation of what had up until then been the preserve of an animallian nervous system.

Artificial intelligence is a badly defined term. Successful deployments of intelligent systems are common, but normally they are redefined to be non-intelligent. My favourite example is the Centrifugal governor. Applied to the Steam Engine by Boulton and Watt and immortalised in the arms of the statue of “Science” on the Holborn viaduct in London, the centrifugal governor automatically regulated the speed of a steam engine, closing the inlet valve progressively as the engine ran faster. It did the job that an intelligent operator used to have to do, but few today would describe it as “artificial intelligence”.

The current revolution in AI is being driven by machine learning. Machine learning is an approach to prediction which is data driven. It is not the first approach to focus on data, the statistical sciences have combined models with data for a number of years. But machine learning has taken a particular focus on improving the quality of prediction, whereas statistical sciences have traditionally focussed more on explaination. Machine learning is giving us information processing engines that are equivalent to the steam engines of the industrial revolution.

## What is Machine Learning? 

Machine learning allows us to extract knowledge from data to form a prediction.

$$\text{data} + \text{model} \xrightarrow{\text{compute}} \text{prediction}$$

A machine learning prediction is made by combining a model with data to form the prediction. The manner in which this is done gives us the machine learning algorithm.

Machine learning models are mathematical models which make weak assumptions about data, e.g. smoothness assumptions. By combining these assumptions with the data, we observe we can interpolate between data points or, occasionally, extrapolate into the future.

Machine learning is a technology which strongly overlaps with the methodology of statistics. From a historical/philosophical view point, machine learning differs from statistics in that the focus in the machine learning community has been primarily on accuracy of prediction, whereas the focus in statistics is typically on the interpretability of a model and/or validating a hypothesis through data collection.

The rapid increase in the availability of compute and data has led to the increased prominence of machine learning. This prominence is surfacing in two different but overlapping domains: data science and artificial intelligence.

## From Model to Decision 

The real challenge, however, is end-to-end decision making. Taking information from the environment and using it to drive decision making to achieve goals.

## Artificial vs Natural Systems 

Let’s take a step back from artificial intelligence, and consider natural intelligence. Or even more generally, let’s consider the contrast between an artificial system and an natural system. The key difference between the two is that artificial systems are designed whereas natural systems are evolved.

Systems design is a major component of all Engineering disciplines. The details differ, but there is a single common theme: achieve your objective with the minimal use of resources to do the job. That provides efficiency. The engineering designer imagines a solution that requires the minimal set of components to achieve the result. A water pump has one route through the pump. That minimises the number of components needed. Redundancy is introduced only in safety critical systems, such as aircraft control systems. Students of biology, however, will be aware that in nature system-redundancy is everywhere. Redundancy leads to robustness. For an organism to survive in an evolving environment it must first be robust, then it can consider how to be efficient. Indeed, organisms that evolve to be too efficient at a particular task, like those that occupy a niche environment, are particularly vulnerable to extinction.

This notion is akin to the idea that only the best will survive, popularly encoded into an notion of evolution by Herbert Spencer’s quote.

Survival of the fittest

Herbet Spencer, 1864

Darwin himself never said “Survival of the Fittest” he talked about evolution by natural selection.

Non-survival of the non-fit

Evolution is better described as “non-survival of the non-fit”. You don’t have to be the fittest to survive, you just need to avoid the pitfalls of life. This is the first priority.

So it is with natural vs artificial intelligences. Any natural intelligence that was not robust to changes in its external environment would not survive, and therefore not reproduce. In contrast the artificial intelligences we produce are designed to be efficient at one specific task: control, computation, playing chess. They are fragile.

The first rule of a natural system is not be intelligent, it is “don’t be stupid”.

A mistake we make in the design of our systems is to equate fitness with the objective function, and to assume it is known and static. In practice, a real environment would have an evolving fitness function which would be unknown at any given time.

You can also read this blog post on Natural and Artificial Intelligence..

The first criterion of a natural intelligence is don’t fail, not because it has a will or intent of its own, but because if it had failed it wouldn’t have stood the test of time. It would no longer exist. In contrast, the mantra for artificial systems is to be more efficient. Our artificial systems are often given a single objective (in machine learning it is encoded in a mathematical function) and they aim to achieve that objective efficiently. These are different characteristics. Even if we wanted to incorporate don’t fail in some form, it is difficult to design for. To design for “don’t fail”, you have to consider every which way in which things can go wrong, if you miss one you fail. These cases are sometimes called corner cases. But in a real, uncontrolled environment, almost everything is a corner. It is difficult to imagine everything that can happen. This is why most of our automated systems operate in controlled environments, for example in a factory, or on a set of rails. Deploying automated systems in an uncontrolled environment requires a different approach to systems design. One that accounts for uncertainty in the environment and is robust to unforeseen circumstances.

## Today’s Artificial Systems

The systems we produce today only work well when their tasks are pigeonholed, bounded in their scope. To achieve robust artificial intelligences we need new approaches to both the design of the individual components, and the combination of components within our AI systems. We need to deal with uncertainty and increase robustness. Today, it is easy to make a fool of an artificial intelligent agent, technology needs to address the challenge of the uncertain environment to achieve robust intelligences.

However, even if we find technological solutions for these challenges, it may be that the essence of human intelligence remains out of reach. It may be that the most quintessential element of our intelligence is defined by limitations. Limitations that computers have never experienced.

Claude Shannon developed the idea of information theory: the mathematics of information. He defined the amount of information we gain when we learn the result of a coin toss as a “bit” of information. A typical computer can communicate with another computer with a billion bits of information per second. Equivalent to a billion coin tosses per second. So how does this compare to us? Well, we can also estimate the amount of information in the English language. Shannon estimated that the average English word contains around 12 bits of information, twelve coin tosses, this means our verbal communication rates are only around the order of tens to hundreds of bits per second. Computers communicate tens of millions of times faster than us, in relative terms we are constrained to a bit of pocket money, while computers are corporate billionaires.

Our intelligence is not an island, it interacts, it infers the goals or intent of others, it predicts our own actions and how we will respond to others. We are social animals, and together we form a communal intelligence that characterises our species. For intelligence to be communal, our ideas to be shared somehow. We need to overcome this bandwidth limitation. The ability to share and collaborate, despite such constrained ability to communicate, characterises us. We must intellectually commune with one another. We cannot communicate all of what we saw, or the details of how we are about to react. Instead, we need a shared understanding. One that allows us to infer each other’s intent through context and a common sense of humanity. This characteristic is so strong that we anthropomorphise any object with which we interact. We apply moods to our cars, our cats, our environment. We seed the weather, volcanoes, trees with intent. Our desire to communicate renders us intellectually animist.

But our limited bandwidth doesn’t constrain us in our imaginations. Our consciousness, our sense of self, allows us to play out different scenarios. To internally observe how our self interacts with others. To learn from an internal simulation of the wider world. Empathy allows us to understand others’ likely responses without having the full detail of their mental state. We can infer their perspective. Self-awareness also allows us to understand our own likely future responses, to look forward in time, play out a scenario. Our brains contain a sense of self and a sense of others. Because our communication cannot be complete it is both contextual and cultural. When driving a car in the UK a flash of the lights at a junction concedes the right of way and invites another road user to proceed, whereas in Italy, the same flash asserts the right of way and warns another road user to remain.

Our main intelligence is our social intelligence, intelligence that is dedicated to overcoming our bandwidth limitation. We are individually complex, but as a society we rely on shared behaviours and oversimplification of our selves to remain coherent.

This nugget of our intelligence seems impossible for a computer to recreate directly, because it is a consequence of our evolutionary history. The computer, on the other hand, was born into a world of data, of high bandwidth communication. It was not there through the genesis of our minds and the cognitive compromises we made are lost to time. To be a truly human intelligence you need to have shared that journey with us.

Of course, none of this prevents us emulating those aspects of human intelligence that we observe in humans. We can form those emulations based on data. But even if an artificial intelligence can emulate humans to a high degree of accuracy it is a different type of intelligence. It is not constrained in the way human intelligence is. You may ask does it matter? Well, it is certainly important to us in many domains that there’s a human pulling the strings. Even in pure commerce it matters: the narrative story behind a product is often as important as the product itself. Handmade goods attract a price premium over factory made. Or alternatively in entertainment: people pay more to go to a live concert than for streaming music over the internet. People will also pay more to go to see a play in the theatre rather than a movie in the cinema.

In many respects I object to the use of the term Artificial Intelligence. It is poorly defined and means different things to different people. But there is one way in which the term is very accurate. The term artificial is appropriate in the same way we can describe a plastic plant as an artificial plant. It is often difficult to pick out from afar whether a plant is artificial or not. A plastic plant can fulfil many of the functions of a natural plant, and plastic plants are more convenient. But they can never replace natural plants.

In the same way, our natural intelligence is an evolved thing of beauty, a consequence of our limitations. Limitations which don’t apply to artificial intelligences and can only be emulated through artificial means. Our natural intelligence, just like our natural landscapes, should be treasured and can never be fully replaced.

## Engineering Systems Design

Engineering systems design is a major component of all engineering disciplines, in each domain the details differ, but there is a common theme. The aim is to achieve your objective with the minimal use of resources to do the job. This provides efficiency. An engineering designer imagines a solution that requires the minimal set of components to achieve a given result.

For example, a water pump has a single route through the pump, the mechanism of the pump involves a single non-redundant set of linkages. This minimizes the resources needed for manufacturing the pump.

## Don’t Fail

The first criterion of natural intelligence is don’t fail. This is in contrast to the mantra for artificial systems which is to be more efficient. Artificial systems are typically given a single objective (in machine learning it is encoded in a mathematical function). The aim is to achieve that objective efficiently.

Even if we wanted to incorporate don’t fail in some form, it is surprisingly difficult to design for. You would have to consider every which way in which things can go wrong. If you miss one you will fail. These cases are sometimes called corner cases.

In an uncontrolled environment there are corners everywhere. It is difficult to imagine everything that can happen. As a result most of our automated systems operate in controlled environments, e.g. in a factory, on a set of rails.

We can imagine a driverless car operating on a highway (a more controlled environment), but it is harder to imagine how one would operate in the city centres of Naples, Kampala or Delhi (less controlled environments).

Deployment in uncontrolled environments may require a different approach to systems design, one that accounts for uncertainty in the environment and one that is robust to unforeseen circumstances.

## Peppercorns 

Asking Siri “What is a trillion to the power of a thousand minus one?” leads to a 30 minute response1 consisting of only 9s. I found this out because my nine year old grabbed my phone and did it. The only way to stop Siri was to force closure. This is an interesting example of a system feature that’s not a bug, in fact it requires clever processing from Wolfram Alpha. But it’s an unexpected result from the system performing correctly.

This challenge of facing a circumstance that was unenvisaged in design but has consequences in deployment becomes far larger when the environment is uncontrolled. Or in the extreme case, where actions of the intelligent system effect the wider environment and change it.

These unforseen circumstances are likely to lead to need for much more efficient turn-around and update for our intelligent systems. Whether we are correcting for security flaws (which are bugs) or unenvisaged circumstantial challenges: an issue I’m referring to as peppercorns. Rapid deployment of system updates is required. For example, Apple have “fixed” the problem of Siri returning long numbers.

The challenge is particularly acute because of the scale at which we can deploy AI solutions. This means when something does go wrong, it may be going wrong in billions of households simultaneously.

You can also check this blog post on Decision Making and Diversity. and this blog post on Natural vs Artifical Intelligence..

## A Man and His Dog 

This picture is from the MARS 2018 conference, it shows Jeff Bezos with SpotMini, Boston Dynamics mini robot dog. It resonates with me in two ways, first of all, despite the life like nature of the dog, it is not using any ‘modern AI’ at all, Marc Raibert is proud of the fact that his robots are using complex cascades of traditional controllers. This allows them to verify performance of the individual controllers which are switched in to different domains.

Classical control emerged from the first wave of work on intelligent systems, a feild known as cybernetics. But it has been redefined to be not an AI technology.

Yet, we can see the aims of control engineering in the field of reinforcement learning. There is large overlap between the fields. A major difference is that reinforcement learning researchers accept the fact that they may be unable to establish guarantees around performance that can be proven mathematically. Mathematica guarantees are a mainstay of the field of control which evolved in an era before large scale compute and data was available.

Secondly, the robot dog is missing a major characteristic of our intelligence, one which real dogs do show, empathy and an ability to model us and who we are. The most complex thing in our environment is other humans, and perhaps the most important facet of our intelligence is our social intelligence. Our ability to model other humans. Something that our real-world companions, our dogs and cats, exhibit better than our artificial companions.

The mythology around intelligence centres on an idea of an intelligence that is better than us in all aspects. But what if some aspects of our intelligence, such as our emotional intelligence, are a consequence of our limitations? What if it is not possible to be emotionally intelligent to the same degree as a human being without experiencing the same limitations as that human being. One obvious limitation is death, but there are other limitations including constraints on our ability to communicate our thoughts.

## The Diving Bell and the Butterfly 

The Diving Bell and the Butterfly is the autobiography of Jean Dominique Bauby. Jean Dominique was the editor of the French Elle magazine, in 1995 at the age of 43, he suffered a major stroke. The stroke paralyzed him and rendered him speechless. He was only able to blink his left eyelid, he became a sufferer of locked in syndrome.

Incredibly, Jean Dominique wrote his book after he became locked in. It took him 10 months of four hours a day to write the book. Each word took two minutes to write.

The idea behind embodiment factors is that we are all in that situation. While not as extreme as for Bauby, we all have somewhat of a locked in intelligence.

## Embodiment Factors 

Let me explain what I mean. Claude Shannon introduced a mathematical concept of information for the purposes of understanding telephone exchanges.

Information has many meanings, but mathematically, Shannon defined a bit of information to be the amount of information you get from tossing a coin.

If I toss a coin, and look at it, I know the answer. You don’t. But if I now tell you the answer I communicate to you 1 bit of information. Shannon defined this as the fundamental unit of information.

If I toss the coin twice, and tell you the result of both tosses, I give you two bits of information. Information is additive.

Shannon also estimated the average information associated with the English language. He estimated that the average information in any word is 12 bits, equivalent to twelve coin tosses.

So every two minutes Bauby was able to communicate 12 bits, or six bits per minute.

This is the information transfer rate he was limited to, the rate at which he could communicate.

Compare this to me, talking now. The average speaker for TEDX speaks around 160 words per minute. That’s 320 times faster than Bauby or around a 2000 bits per minute. 2000 coin tosses per minute.

But, just think how much thought Bauby was putting into every sentence. Imagine how carefully chosen each of his words was. Because he was communication constrained he could put more thought into each of his words. Into thinking about his audience.

So, his intelligence became locked in. He thinks as fast as any of us, but can communicate slower. Like the tree falling in the woods with no one there to hear it, his intelligence is embedded inside him.

Two thousand coin tosses per minute sounds pretty impressive, but this talk is not just about us, it’s about our computers, and the type of intelligence we are creating within them.

So how does two thousand compare to our digital companions? When computers talk to each other, they do so with billions of coin tosses per minute.

Let’s imagine for a moment, that instead of talking about communication of information, we are actually talking about money. Bauby would have 6 dollars. I would have 2000 dollars, and my computer has billions of dollars.

The internet has interconnected computers and equipped them with extremely high transfer rates.

However, by our very best estimates, computers actually think slower than us.

How can that be? You might ask, computers calculate much faster than me. That’s true, but underlying your conscious thoughts there are a lot of calculations going on.

Each thought involves many thousands, millions or billions of calculations. How many exactly, we don’t know yet, because we don’t know how the brain turns calculations into thoughts.

Our best estimates suggest that to simulate your brain a computer would have to be as large as the UK Met Office machine here in Exeter. That’s a 250 million pound machine, the fastest in the UK. It can do 16 billion billon calculations per second.

It simulates the weather across the word every day, that’s how much power we think we need to simulate our brains.

So, in terms of our computational power we are extraordinary, but in terms of our ability to explain ourselves, just like Bauby, we are locked in.

For a typical computer, to communicate everything it computes in one second, it would only take it a couple of minutes. For us to do the same would take 15 billion years.

If intelligence is fundamentally about processing and sharing of information. This gives us a fundamental constraint on human intelligence that dictates its nature.

I call this ratio between the time it takes to compute something, and the time it takes to say it, the embodiment factor (Lawrence 2017). Because it reflects how embodied our cognition is.

If it takes you two minutes to say the thing you have thought in a second, then you are a computer. If it takes you 15 billion years, then you are a human.

But in terms of our ability to deploy that computation in actual use, to share the results of what we have inferred, we are very limited. So when you imagine the F1 car that represents a psyche, think of an F1 car with bicycle wheels.

Just think of the control a driver would have to have to deploy such power through such a narrow channel of traction. That is the beauty and the skill of the human mind.

In contrast, our computers are more like go-karts. Underpowered, but with well-matched tires. They can communicate far more fluidly. They are more efficient, but somehow less extraordinary, less beautiful.

## Human Communication 

For human conversation to work, we require an internal model of who we are speaking to. We model each other, and combine our sense of who they are, who they think we are, and what has been said. This is our approach to dealing with the limited bandwidth connection we have. Empathy and understanding of intent. Mental dispositional concepts are used to augment our limited communication bandwidth.

Fritz Heider referred to the important point of a conversation as being that they are happenings that are “psychologically represented in each of the participants” (his emphasis) (Heider 1958)

### Bandwidth Constrained Conversations

import pods
from ipywidgets import IntSlider
pods.notebook.display_plots('anne-bob-conversation{sample:0>3}.svg',
'../slides/diagrams',  sample=IntSlider(0, 0, 7, 1))

Embodiment factors imply that, in our communication between humans, what is not said is, perhaps, more important than what is said. To communicate with each other we need to have a model of who each of us are.

To aid this, in society, we are required to perform roles. Whether as a parent, a teacher, an employee or a boss. Each of these roles requires that we conform to certain standards of behaviour to facilitate communication between ourselves.

Control of self is vitally important to these communications.

The high availability of data available to humans undermines human-to-human communication channels by providing new routes to undermining our control of self.

### A Six Word Novel 

But this is a very different kind of intelligence than ours. A computer cannot understand the depth of the Ernest Hemingway’s apocryphal six word novel: “For Sale, Baby Shoes, Never worn”, because it isn’t equipped with that ability to model the complexity of humanity that underlies that statement.

## Computer Conversations 

import pods
from ipywidgets import IntSlider
pods.notebook.display_plots('anne-bob-conversation{sample:0>3}.svg',
'../slides/diagrams',  sample=IntSlider(0, 0, 7, 1))

Similarly, we find it difficult to comprehend how computers are making decisions. Because they do so with more data than we can possibly imagine.

In many respects, this is not a problem, it’s a good thing. Computers and us are good at different things. But when we interact with a computer, when it acts in a different way to us, we need to remember why.

Just as the first step to getting along with other humans is understanding other humans, so it needs to be with getting along with our computers.

Embodiment factors explain why, at the same time, computers are so impressive in simulating our weather, but so poor at predicting our moods. Our complexity is greater than that of our weather, and each of us is tuned to read and respond to one another.

Their intelligence is different. It is based on very large quantities of data that we cannot absorb. Our computers don’t have a complex internal model of who we are. They don’t understand the human condition. They are not tuned to respond to us as we are to each other.

Embodiment factors encapsulate a profound thing about the nature of humans. Our locked in intelligence means that we are striving to communicate, so we put a lot of thought into what we’re communicating with. And if we’re communicating with something complex, we naturally anthropomorphize them.

We give our dogs, our cats and our cars human motivations. We do the same with our computers. We anthropomorphize them. We assume that they have the same objectives as us and the same constraints. They don’t.

This means, that when we worry about artificial intelligence, we worry about the wrong things. We fear computers that behave like more powerful versions of ourselves that will struggle to outcompete us.

In reality, the challenge is that our computers cannot be human enough. They cannot understand us with the depth we understand one another. They drop below our cognitive radar and operate outside our mental models.

The real danger is that computers don’t anthropomorphize. They’ll make decisions in isolation from us without our supervision, because they can’t communicate truly and deeply with us.

## Transhumanism 

Some researchers talk about transhumanism, releasing us from our own limitations, gaining the bandwidth of the computer. Who wouldn’t want the equivalent of billions of dollars of communication that a computer has?

But what if that would destroy the very nature of what it is to be human. What if we are defined by our limitations. What if our consciousness is born out of a need to understand and be understood by others? What if that thing that we value the most is a side effect of our limitations?

AI is a technology, it is not a human being. It doesn’t worry it is being misunderstood, it doesn’t hate us, it doesn’t love us, it doesn’t even have an opinion about us.

Any sense that it does is in that little internal model we have as we anthropomorphize it. AI doesn’t stand for anthropomorphic intelligence, it stands for artificial intelligence. Artificial in the way a plastic plant is artificial.

Of course, like any technology, that doesn’t mean it’s without its dangers. Technological advance has always brought social challenges and likely always will, but if we are to face those challenges head on, we need to acknowledge the difference between our intelligence and that which we create in our computers.

Your cat understands you better than your computer does, your cat understands me better than your computer does, and it hasn’t even met me!

Our lives are defined by our desperate need to be understood: art, music, literature, dance, sport. So many creative ways to try and communicate who we are or what we feel. The computer has no need for this.

When you hear the term Artificial Intelligence, remember that’s precisely what it is. Artificial, like that plastic plant. A plastic plant is convenient, it doesn’t need watering, it doesn’t need to be put in the light, it won’t wilt if you don’t attend to it, and it won’t outgrow the place you put it.

A plastic plant will do some of the jobs that a real plant does, but it isn’t a proper replacement, and never will be. So, it is with our artificial intelligences.

I believe that our fascination with AI is actually a projected fascination with ourselves. A sort of technological narcissism. One of the reasons that the next generation of artificial intelligence solutions excites me is because I think it will lead to a much better understanding of our own intelligence.

But with our self-idolization comes a Icarian fear of what it might mean to emulate those characteristics that we perceive of as uniquely human.

Our fears of AI singularities and superintelligences come from a confused mixing of what we create and what we are.

Do not fool yourselves into thinking these computers are the same thing as us, they never will be. We are a consequence of our limitations, just as Bauby was defined by his. Or maybe limitations is the wrong word, as Bauby described there are always moments when we can explore our inner self and escape into our own imagination:

My cocoon becomes less oppressive, and my mind takes flight like a butterfly. There is so much to do. You can wander off in space or in time, set out for Tierra del Fuego or King Midas’s court. You can visit the woman you love, slide down beside her and stroke her still-sleeping face. You can build castles in Spain, steal the golden fleece, discover Atlantis, realize your childhood dreams and adult ambitions.
Enough rambling. My main task now is to compose the first of those bedridden travel notes so that I shall be ready when my publisher’s emissary arrives to take my dictation, letter by letter. In my head I churn over every sentence ten times, delete a word, add an adjective, and learn my text by heart, paragraph by paragraph.

The flower that is this book, that is this fight, can never bud from an artificial plant.

# Conclusions

• We are a long way from emulating human intelligence, animal intelligence, animal motion.
• The objectives of cybernetics still have not been reached.
• The robustness of natural systems is outside the scope of our current design methodologies.

• There is something quintisential about the human experience.
• We are co-evolved to view the world in a certain way to enable collaboration.
• Our consciousness is a consequence of our limitations. Our locked-in intelligence.

# References

Heider, Fritz. 1958. The Psychology of Interpersonal Relations. John Wiley.

Lawrence, Neil D. 2017. “Living Together: Mind and Machine Intelligence.” arXiv. https://arxiv.org/abs/1705.07996.

1. Apple has fixed this issue so that Siri no longer does this.