Entropy, ambiguity, and the evolution of the human mind

Darwin, when he wanted to test his own theory to breaking point, reached for the eye. An organ so intricate, he admitted, that to suppose it had formed by natural selection seemed absurd in the highest degree. It was not absurd, because biology can trace the eye back, step by patient step, to a single light-sensitive cell, through a long series of intermediate forms, each conferring a slightly better chance of survival than the last: a patch of pigment, a shallow cup, a pinhole, a lens. There is no jump anywhere in the sequence, no moment at which an eye springs into being. There is only accumulation, each small improvement paid for by the survival it bought.

The thought that has organised the last several years of my work follows from that and would not let me rest once I had it. If evolution built the eye by increments from a single cell, it built cognition the same way, and there must be, somewhere at the beginning, a photosensitive cell for thought, a minimal precursor from which the whole apparatus of a human mind was assembled under the same slow pressure. Whatever a mind is, it did not arrive whole. There is an unbroken chain of increasingly capable minds running back from the one reading this sentence to the first living thing that did anything we would recognise as cognition, and if that chain is real then the question of what a mind is becomes the question of what pressure built it. The answer Ken Wenger and I have arrived at is that the pressure was not, in the main, the complexity of the world. It was the two faces of information that no amount of learning resolves: entropy, the part of the world that cannot be predicted, and ambiguity, the part that has already been decided and cannot be read. A mind is what those two pressures made, and the making has a shape that can be traced.

Life Against the Current

A living thing is a pocket of order in a universe that runs towards disorder. Left alone, everything tends to even out, to cool, to scatter, to lose its structure, and that is the meaning of entropy in physics, the slide from arrangement into sameness. A rock submits to it. A living thing holds it off, maintaining its form, repairing itself, keeping its inside different from its outside, and doing so continuously against a current that never stops. Life is entropy resistance, and every organism that has ever existed is engaged in it. Schrödinger said as much in 1944, in What is Life?, where he wrote that an organism feeds on negative entropy; the phrase has been argued over ever since, and the observation has not.

Two words are doing work in that paragraph and they are cousins rather than twins. The entropy of physics is disorder in the world. The entropy Ken and I mean when we talk about cognition is uncertainty relative to a model, the mark of where a system’s understanding of the world gives out. They are not the same quantity. The connection is that a living thing must hold off both at once. It keeps its form against the first by reading the second, and the only question that turns out to matter, the one that organises the whole story of cognition, is how effectively a given creature reads, and across how many kinds of uncertainty.

Consider the simplest reading there is. A euglena in a pond rises towards the light when the light is there and sinks when it is gone. That is not intelligence, and the behaviour is written into its chemistry rather than chosen, but look at what it contains. There is a model, if the word can be stretched that far, an expectation about where energy will be. There is a sensor that detects a departure from the expectation, the light failing. There is a change of behaviour triggered by the departure. A bacterium does something a little richer. Swimming up a chemical gradient that is improving, it runs; when the gradient stops improving, it tumbles, randomising its direction, and tries again. The tumble is the interesting part. It is not a better reading of the gradient. It is the creature detecting that its current mode is no longer working and switching to another, carried out by a system far too simple to be doing anything we would call inference. That is the photosensitive cell of cognition. It already has the two-part structure that every mind since has elaborated: something that acts on a model of the world, and something that notices when the model has stopped working. The architecture is not a late human achievement. It is present at the floor of life, in a creature with no brain, because resisting entropy in an uncertain world requires it from the very start.

What Cannot Be Optimised Away

The temptation, once you see the two-part structure, is to treat the second part as a temporary embarrassment, a crude fix for a model that a better creature would simply improve. That is the assumption the whole of modern artificial intelligence rests on, and it is worth stating what is wrong with it, because the same mistake distorts our picture of how minds evolved.

Information, seen from the side of a creature that must act on it rather than merely carry it, has three faces, complexity, entropy and ambiguity, which I have set out in the primer The Three Faces of Information. What matters here is what a creature does when it mistakes one face for another. A creature that treats entropy as a harder case of complexity will keep looking for a pattern that is not there, and because finding patterns is what its optimiser does, it will find one anyway, and be confidently wrong. A creature that treats ambiguity that way will collapse a real fork into a single guess and never know a choice was made. Neither error can be trained out, because the missing information was never in any data. Ken and I have shown, for machines, that this blindness is a provable consequence of optimising a single objective, and the proof does not care whether the optimiser is made of silicon or protein.

That is why the second part of the bacterium is not a crude fix. It is the only possible response to two faces of information that no first part, however sophisticated, can reduce. Where the world is complexity, a better model helps. Where the world is entropy or ambiguity, no model helps, and the only thing a creature can do is notice that it has reached the edge of what it can resolve and change what it is doing. Evolution does not build perfect systems. It builds effective ones, and the effective response to the unresolvable is not to resolve it but to govern it: to hedge, to hold more than one possibility open, to gather one more piece of evidence, to switch mode, to flee. The governance of uncertainty, not its elimination, is what selection has been paying for since the first cell that could tell that the light had gone.

The nearest neighbour to this account is Karl Friston’s free-energy principle, and I should say where the two part. Friston’s organism also acts on a model and minimises the surprise the model registers, and in its active-inference form it will even seek out the surprise that reduces future surprise, which is the nearest thing in the literature to the opportunity reading I come to below. The difference is what the signal is for. In Friston’s account the signal exists to be driven down; the whole organism is one minimiser. In the account Ken and I have built, the signal is read before it is reduced, and read as one of three different messages, because a creature that only minimises will minimise its way past a fork it should have held open. Whether that is a disagreement or a refinement is the argument I would most like to have with him.

The Signal

Something follows from this that reframes what uncertainty is for, and it arrived for me as an intuition long before either of us could defend it. You have had the feeling. You are driving a road you have driven a hundred times and something is wrong before you can say what. A prickle at the back of the neck, a sense that the scene does not fit, and you lift your foot from the accelerator, and only afterwards, if ever, do you work out what your mind had caught, the parked car with someone sitting too still in it, the quality of the light before a storm. The feeling arrived before the explanation. What reached you first was not information about the world but information about your own understanding of it, a flag that said your model of this scene has sprung a leak, and you treated the flag as valuable.

That flag is entropy, in the cognitive sense, and it is the most useful thing you felt all day. A physician who has seen ten thousand patients will sometimes look at one and feel, before any test, that the case does not fit. She is not being mystical. Her training has built a detailed model of an ordinary presentation, and the feeling is that model registering a departure from itself. The novice does not have the feeling, not because he is calmer but because he has no model precise enough to be violated. The capacity to be usefully uncertain is not a weakness that expertise removes. It is a capacity that expertise sharpens, because a better model produces a cleaner signal when the world departs from it.

Almost every objective function in machine learning is a mechanism for driving that signal down, and the exceptions, the curiosity-driven methods that pay a system for seeking surprise, make the point by having to be bolted on as a second term. Within a closed problem the driving down is exactly right. In an open world it is committed to an impossible project, and worse, it treats as failure the single most decision-relevant fact a system can have, which is the location of the boundary between where it can trust itself and where it cannot. A mind does the opposite. It reads the signal, and it reads it as one of three messages, because a spike of uncertainty does not always say the same thing. It can mean danger, the situation has moved outside anything the creature can handle, and the right response is restraint. It can mean drift, the world has shifted a little and a small adjustment will serve. Or it can mean opportunity, a persistent, structured surprise that says there is something real here the creature’s picture of the world does not contain, and the right response is to investigate and, perhaps, to rebuild. A creature that collapses the three into one is either paralysed by every novelty or reckless before every threat. The skill is in the distinction, and the distinction is where the story of the mind’s growth begins, because a creature that can tell the three apart has something to grow.

The Enemy Within

For a long stretch of the story the uncertainty a creature faces comes entirely from outside it. The temperature shifts, the gradient wavers, a predator moves, the food is here this season and gone the next. Governance evolves to meet it, in better sensors, faster reflexes, a wider repertoire of responses. This is the first long age of cognition, and a great deal of life never leaves it. The uncertainty is exogenous, and the mind grows by adding parts that handle more of the outside world.

Then something changes, and it is the pivot of the whole account. As a creature’s mind deepens, as it gains memory that persists, predictions that reach into the future, models of what other creatures are about to do, the internal machinery becomes complex enough to generate uncertainty of its own. A mind that can remember can hold two memories that point to incompatible conclusions. A mind that can predict can run two models that disagree. A mind that can plan can find two courses of action equally poised. A mind that can model other minds falls into the hall of mirrors of modelling a creature that is modelling it, uncertainty compounding through second-guessing without any change in the world at all. This is endogenous entropy, uncertainty the system generates from inside, and it is not a malfunction. It is the unavoidable price of a richer model. The very depth that let the creature master the outside world turns the creature into a source of uncertainty to itself. The enemy, having been outside for a billion years, moves in.

It is worth being concrete about what that feels like, because it is most of your inner life. The ache of a decision you cannot settle is two parts of you generating incompatible recommendations that neither can overrule. The grip of a memory that will not fit the story you tell about yourself is stored experience yielding incompatible predictions about who you are. The vertigo of wondering what someone thinks of you, and what they think you think of them, is social modelling folding back on itself. None of these is caused by anything happening in the room. They are the sound of a deep mind generating uncertainty about its own states, and a great deal of what we call the examined life, and nearly all of what we call worry, is the work of governing an entropy that has no source outside the mind that suffers it. Here, too, the body plays its part. The chemistry that regulates mood, the electrical weather of billions of neurons, the signals of hunger and fatigue and fear, are inputs the mind did not ask for and cannot switch off, and a system optimised to ignore noise would have to ignore them. A mind cannot, because they are not noise. They are the creature’s own state, arriving as uncertainty, and they must be governed like everything else.

Every Solution a New Problem

Here is the mechanism that drives the whole ascent, and once you see it you see it everywhere. Entropy, in a cognitive system, is not eliminated by being governed. It is transformed. A mechanism that reduces uncertainty in one domain does so by adding machinery, and that machinery, being complex, generates new uncertainty in another. The solution to one problem is the cause of the next. A solitary creature exposed to predators reduces that danger by banding into a group, and the physical uncertainty falls, but now it must coordinate with others, read their intentions, negotiate rank and share, and a whole new domain of social uncertainty has opened where none existed. That step is the one Nicholas Humphrey proposed in 1976 and Robin Dunbar later gave numbers to, the social brain: the claim that primate intelligence was built by the demands of living with other primates rather than by the demands of the forest. I take it as established, and as one step. What the account here adds is that the social step is not special. It is one instance of a general rule, that every governor manufactures the problem the next one will be built for, and the same rule has to run through the temporal and symbolic steps that the social brain does not reach. A creature plagued by an unpredictable environment evolves memory, and the ecological uncertainty falls, but memory lets it project forward as well as back, and now it can anticipate, which means it can dread, and a new domain of temporal uncertainty has opened, the uncertainty of a future it can imagine but not control. A group whose coordination is unstable invents norms to steady it, and the social uncertainty falls, but norms are symbols, and symbols can be read more than one way, and a new domain of symbolic uncertainty has opened in turn.

This is a ratchet rather than a ladder. Physical to social to temporal to symbolic, the dominant uncertainty a lineage faces migrates from domain to domain, never disappearing, only changing address, and the mind thickens at each step to meet it where it has moved. I find this the most astonishing idea in the whole theory, because it explains something no other account does. It explains why cognition kept going. If minds evolved merely to solve the problems the world posed, they should have stopped when the world’s problems were solved. They did not stop, and this is why. The problems that drove the later stages of the ascent were not the world’s. They were the ones our own solutions had manufactured. We think, in large part, because thinking made us into creatures with more to think about.

One consequence corrects a picture almost everyone carries. We rank minds on a single line, from simple to complex, with ourselves at the top and everything else arranged by how close it comes to us. The entropy account says there is no single line. A creature is shaped by the particular domains of uncertainty its way of life presses hardest, and different lives press different domains. A social bird navigates a web of alliances and grudges that would tax a person, and has built its intelligence around that. A solitary hunter has built its own around problems of stalking and escape that have nothing to do with the social uncertainties that shaped us. Neither is a lesser version of the human mind or a step on the way to it. Intelligence is not a height. It is a shape, the shape of the particular entropy a lineage had to govern, and the human shape is not the summit of a staircase but one especially social, especially symbolic pattern among many.

The Recursive Self

Now the step that turns governance into a self. Imagine a creature that detects only two things, the presence of food and its absence. It needs no inner life; it reacts. Now imagine a creature that predicts where the food will be. When the prediction holds, the uncertainty resolves and the system settles. When it fails, the creature must reprocess, and the failure lingers as a state that colours what it does next. That lingering is already more than the bacterium has. Its failure leaves no mark; the tumble happens and is gone. Take one more step. Imagine a creature whose failed prediction becomes available beyond the part of it that made the prediction, readable by the rest of the system, able to change what other parts do. Only here is there a web to speak of, and only here can the creature begin to predict its own internal state, to predict, in the end, its own ability to predict.

Three deepening levels, then, rather than one mechanism stamped out at every scale. At the first a unit does its job and fails when the world exceeds it, with nothing left over. At the second the failure leaves a mark. At the third the mark travels, and the system as a whole can act on it. Continuity from bacterium to person is that criterion strengthening step by step, and where the second and third levels first appear in the tree of life is a matter for evidence rather than assertion.

What happens when the third level deepens is the origin of the self, and it is a mechanism rather than a mystery. A system that models the world needs no model of itself. A system that must govern the uncertainty it generates about itself has no choice but to build one, because there is nothing else against which its own conflicting states can be assessed. Self-awareness, on this account, is not the foundation of cognition. It is what cognition becomes when it is forced to turn its own instruments inward, and it was forced there by the ratchet: the mind that had solved the world’s problems and manufactured its own had, in the end, to keep track of itself to cope. The perception of time comes from the same place. A system that is constantly predicting and never perfectly resolving experiences a continuous state of change, the ordering of the moments when expectation is violated and control has to be reconsidered. A system to which nothing is ever surprising has, in a meaningful sense, no experience of time passing at all, which is why the machines we build, however fluent, live in an eternal present.

This answers a question that has occupied a great deal of philosophy, though only in part, and the part should be named. Why a mind should model itself, and why self-modelling should pay, is what David Chalmers called an easy problem: a question about function, answerable by a mechanism, and the answer above is a mechanism. Why there should be something it is like to run that mechanism, the hard problem, is not answered here, and I do not think the entropy account touches it. What the account removes is the assumption that self-awareness needed a special cause. We are self-aware because evolution kept finding an advantage in deeper minds, and kept finding it until the capacity we had built was flooded, not by the uncertainty of the world outside, but by layer upon layer of uncertainty we were generating about ourselves. Modelling ourselves was the only way to govern the flood. That is the whole of the functional story, and it is the only story this essay claims.

The Web Can Shrink

It would be easy to tell this as a story of inevitable ascent, and the theory forbids it. The growth of a mind is not guaranteed and not one-directional. A new governor is expensive. It costs energy, structure, and the endogenous entropy its own complexity generates, and it is selected only when the uncertainty it relieves is greater than the cost of maintaining it. The arrow can point the other way. When a lineage moves into a world where uncertainty contracts, the machinery built to govern it stops paying for itself, and it is lost. Parasites that settle into the stable, nutrient-rich interior of a host shed, over generations, much of the apparatus their free-living ancestors needed. Fish that colonise lightless caves lose their eyes. Animals we have domesticated, relieved by us of many of the uncertainties their wild cousins face, tend to smaller brains than those cousins. Cognition tracks entropy pressure in both directions, expanding when uncertainty mounts and thinning when it recedes.

I dwell on this because it is what makes the account a scientific claim rather than a flattering myth. A theory that predicts only that minds grow more complex can never be wrong. A theory that predicts minds should simplify exactly when their entropy burden falls has staked something, can be checked against the cave fish and the tapeworm and the domestic dog, and would be embarrassed if cognition grew regardless of pressure. It is not embarrassed, because the pattern holds. It also carries a warning that reaches beyond biology. If cognitive richness is sustained by entropy pressure, then a creature, or a culture, that succeeds in removing all the uncertainty from its world should not expect to keep the mind that the uncertainty built.

The whole evolutionary story deserves its label. That the eye evolved by increments is established beyond serious doubt, and that cognition did too is scarcely more controversial. The specific account here, that the engine of cognitive growth is the migration of uncertainty from domain to domain, and that solutions manufacture the problems that drive the next advance, is a hypothesis. I hold it because it explains a great deal economically and because it makes predictions that could show it false. There is a sharp one. The account rests on a failure signal that travels beyond the part of the mind that raised it, so if careful work showed that a local failure in a capable animal reliably stayed local, changing nothing elsewhere in the system, the mechanism at the centre of the theory would be in serious trouble. It has not been shown, and the comparative evidence runs the other way. Give a monkey or a dolphin a hard perceptual judgement and a third option, a way of declining the trial for a smaller but certain reward, and it uses the option selectively, on exactly the trials that sit closest to its own threshold. Rats, pigeons and crows do it too in some designs. The animals are reporting on the state of their own evidence, which is precisely the capacity this essay has been tracing, and whether that reporting requires anything we would call metacognition or reduces to trained reinforcement is a dispute that has run for 20 years and is not finished. It is the same dispute, in a different species, that the field is having about machines.

The Rationalisation of the Unrationalisable

The phrase in the title means this. Human intelligence is the capacity to rationalise the unrationalisable, and I mean that mechanically rather than as a compliment. It is the capacity to meet information that cannot be predicted and cannot be decided, to read the failure of one’s own model as a signal, and to build structure that holds the unresolved rather than pretending to resolve it. The human mind is the least optimised system in nature and the most governed, and every distinctively human thing follows from that. Culture, on this reading, is governance at the scale of a whole people: moral codes, laws, religions, the rituals that synchronise a community and the stories that give a life a beginning and an end are machinery for stabilising the deepest and longest-horizon uncertainties a mind can generate, the ones no reflex resolves and no single mind holds alone. I make no claim about whether any of it is true. I claim only that its function is to govern an entropy that would otherwise overwhelm the creature that generates it.

We are not, on this account, the creatures who conquered uncertainty. We are the creatures who became most entangled in the uncertainty of our own making. The animal that most thoroughly solved the problems the world set it is the animal that generated the richest new problems from its own solutions, until it found itself lying awake at night with fears no other creature can have, about a death it can foresee, about the opinions of people not in the room, about the meaning of words and the shape of a life. Those are not the failures of a mind that ought to have been more at peace. They are what a mind of this shape does with the entropy it makes. I lie awake with them like everyone else. The account above is the best explanation I have for why.