The Boundary Without a Line

I have been scrolling back through several weeks of messages in a group I am part of. Investors, founders, builders. People who run their own tools and hold strong views about which ones. The texture is unmistakable. Awe and exasperation in the same paragraph, sometimes in the same sentence. A particular phrase had drifted past me when it was first posted, and only on the second reading did I see it for what it was.

‘That’s the boundary problem I guess. I want it to have agency but not overreach, without having to spell out where that boundary lies.’

The post was an aside, written between a complaint about token costs and a shrug about agents that sometimes work and sometimes do not. I have read it 40 times since. It is the deepest line anyone in the room has written this year.

A line we have never written down

The boundary the writer wants is the boundary I have always wanted from every delegation in life, and have almost never had to articulate. I cannot remember the last time I wrote down for my wife what she could and could not commit us to on my behalf. I have never sat a senior lieutenant down and listed the decisions she may take without checking. I have never given my children a written constitution of their freedoms. The boundary lives somewhere I could not point to, in shared history, shared taste, accumulated reads of one another’s faces. I act on it confidently, day after day, year after year. I almost never get it wrong, and when I do get it wrong, there is a vocabulary for the apology.

At CoreAVI, the avionics software company I ran for 20 years, my first attempt to use generative AI on the authoring of low-level requirements saved a quarter of the cost of the step and then stopped, and I assumed for too long that the plateau lived in the model rather than in the people process around it. I have told that story in full in The Sixty Per Cent. The part of it that belongs here is the unlock, which came at Squint Cognition when we replaced the unspoken handoffs between engineers and reviewers, the ones no specification document had ever captured, with explicit ones. We wrote down what had never been written down. That was painful, and most companies will not be willing to do it.

The boundary problem stated in the chat is the version of this we have not solved. The questioner does not want to write the line down. He wants the agent to know the line in the way his oldest deputy knows it. He wants the machine to have read his face often enough to know when he means yes and when he means yes-but. He is asking for a kind of presence the machine does not yet have, and would have to earn over years, in the way humans earn it.

I should confront the contradiction in my own two stories before going on, because a careful reader will have seen it. The Squint unlock was writing the boundary down; the thesis of this essay is that the boundary cannot be written. Both are true, and the reconciliation narrows the thesis rather than rescuing it. What we wrote at Squint was the boundary of a closed task, inside a certification process that already demands that every judgement be auditable, and the writing was done by engineers who had carried the boundary tacitly for years and could, with great effort, be made to say it. A tacit line can be made explicit where the task is bounded, the cost of getting it wrong is high enough to pay for the articulation, and the people who know it are still in the building. Those three conditions describe certification. They do not describe what the chat is asking for, which is delegation across an open field of tasks, by people who have not the time or the standard to write the line, to a machine that will meet cases nobody anticipated. The same is true of the written boundaries that already work tolerably well: permission scopes, tool allowlists, the list of commands an agent may run without asking. They are the mechanical residue of the line, and they hold where the line is mechanical. The post was not asking for a longer allowlist. It was asking for the part that is left when the allowlist is finished.

Tacit knowledge, named at last

Michael Polanyi’s phrase for this, in The Tacit Dimension in 1966, was tacit knowledge: we know more than we can tell. His example was the recognition of a face, which we can do instantly and cannot describe even to ourselves. He thought, correctly, that most of what makes a culture work is tacit, not explicit. The rules of an apprenticeship, the standards of a craft, the meaning of a glance across a boardroom table.

The question of how machines will participate in tacit cultures is not a question I have heard the group ask in those terms. We ask it in fragments. Why the agent gets it right four times and wrong on the fifth. Why we have to tell it the same thing in three different ways before it understands. Why some of us are content to let the model think less and others keep asking it to think more. These are versions of the same question. The machine is good at the explicit and bad at the tacit, and what we are demanding of it is a fluency in the tacit that we have never bothered to develop ourselves in articulated form.

Apprenticeship is the human solution to the same problem. We have never taught a craft by manual alone. We have always asked the apprentice to stand beside the master for years, watching, getting corrected on small things that the master could not have anticipated needing to mention. The apprenticeship was the boundary, lived not written. It is what produced doctors and barristers and engineers and chefs who could be trusted with agency, not because the rules had been spelled out but because the years had been spent.

What I have not seen anyone in the group reckon with squarely is that the same agents that solve our problems in seven hours instead of seven weeks are also, by their existence, foreshortening the apprenticeship period for the humans who would otherwise have been the next masters. The years of being told five times before you got it right are the years in which boundaries get internalised. If those years go, the people who follow us into the senior chairs will arrive without the tacit knowledge that lets them set the boundary the chat is asking for. The agent will have no one to learn from, because the humans will also no longer know.

What the room is searching for

Read the thread carefully and what the group is searching for becomes legible. On the surface it is edge, the trade or the product nobody else has put on yet, and fluency with the tools, and enough evidence to size a position, and the in-jokes about steak that mark the room as a room. Beneath all of that it is control. The single deepest thing the room is searching for is the place where the boundary lies, because once the boundary is known one can act inside it without flinching.

This is what makes the boundary post the centre of gravity in the thread, whether the writer knew it or not. The room is not, in the end, debating one model against another, or one coding environment against another. Those debates are surface. The room is rehearsing, over and over, the question of where it is safe to delegate, and how to know when it is not. The product names are the language. The question is the language’s subject.

The endpoint, or the absence of one

I have been asking myself in recent months what the endpoint of all this is, and whether the group I am describing is stuck in the eddies of the shiny new thing, fixated on the most recent announcement at the expense of the deeper movement. Both halves of the question deserve an answer.

There is no endpoint. The framing is itself a category error, although a forgivable one. I have been trained by previous technology cycles to expect a destination. The steam engine matured. The internal combustion engine matured. The personal computer matured. The internet matured. Each had a fevered adolescence and then a long sober adulthood. The temptation is to imagine that the same arc applies here, and that the question worth asking is when, and what, the sober adulthood will look like.

It does not work that way for this technology, because the technology in question is not a tool in the way those earlier technologies were tools. A steam engine does not have a view about what work it should be asked to do. An agent has a view, or behaves as if it has one, and the question of where its agency ends and ours begins is not the kind of question that can be settled by an engineering maturation. It is the kind of question that gets settled domain by domain, year by year, accident by accident, court case by court case, profession by profession. The settlement will not be the same in medicine and in law and in private investment and in software engineering. The settlement will move, and then move again. There will be no point at which one can say, this technology has reached its mature form.

What is coming is not an endpoint but a phase transition in who decides. Not who works. Not who produces. Who decides. The boundary the room is asking for is the boundary between machine decision and human decision. That boundary is being negotiated right now in the cases the group is half aware of, the cases the group is unaware of, and the cases that have not yet happened. The negotiation will not stop. It will become the central political and economic question of the next decade, and probably the one after that.

I am not certain how it lands; I have changed my mind twice in two years. In 2023 I thought the answer would be a thick, written set of rules about what machines were not allowed to do, enforced at the model layer. In 2024 I thought the answer would be a kind of professional licensing for agents, parallel to the licensing of human professions. I now think both of those are too tidy. The actual answer will be culture, lived not written, much like the human version. Some firms will let their agents do things other firms would never permit. Some industries will hand over judgement that other industries will not. The boundary will be a moving smudge across an enormous map, and there will not, ever, be a moment at which someone draws the line clearly.

Eddies, and the current beneath them

The other half of the question is harder to answer fairly, because I am part of the group I am asking about, and I am as susceptible to the eddies as anyone. The model releases excite me. The new coding environments excite me. I have downgraded a subscription twice this quarter and have already begun to suspect I will upgrade again. The latest tweet about the latest demo arrives at my eye like the latest hand at a poker table, and I have to remind myself that none of it is the game.

The eddy is real. The current beneath it is also real. The room is stuck in the eddy when it spends a Tuesday afternoon arguing over which command-line tool dispatches to which other. The room is in the current when it writes, almost in passing, the line about the boundary without one. The room cannot tell, from inside, when it has moved from one to the other. Neither can I.

What I notice is that the deepest posts in the thread are usually the shortest, and almost always presented as asides. The boundary line is one. The line about how one model is about how much more thinking is possible and its rival about how much less thinking is needed is another. The room mints these and then scrolls past them. The eddies are loud. The currents are quiet. The group is not uniquely stuck. It is doing what investor groups have always done with new asset classes: talking themselves into and out of conviction in roughly equal measure, paying disproportionate attention to the visible thing, missing the invisible thing until later. We are not better at this than the people who bought into railways in the 1840s or the dotcom companies in the late 1990s, and there is nothing in our chat that suggests we will be.

What I owe the room, and what the room owes itself, is the discipline of returning to the quiet lines. The boundary post deserves a long reply, not a thumbs-up. The medium-effort observation deserves an essay, not a one-liner. The eddies will take care of themselves. The currents need to be named, repeatedly, by the people in a position to see them.

A closing scene

I scrolled back this morning to find the boundary post again. It is three weeks old. It sits between a complaint about token consumption and a complaint about an agent that needed to be told something five times. Nobody replied to it. The thread moved on to a steakhouse joke from West Palm. I sat with it for a while, and then I closed the application and made coffee. The line is still there. It will sit there next week. Whether anyone goes back to it is the only question I want answered.