AI has been explored arguably before any of us were born. Back in my student days, we used to picture how AI might eventually take shape. Possibly as expert systems etc. If we dig through the old posts on my blogs, we’ll find that what I once imagined AI to be is rather different from what it has actually become. In those days, I pictured AI emerging from one of three things:
- Strange loop. This was taken after Hofstadter’s idea that the “self” is a self-reference that forms once a system becomes complex enough to hold a model of itself within itself. Consciousness, on this account, emerges from that.
- Complexity and its emergence. I imagined consciousness as a property of systems organised far from equilibrium. Per Bak showed that such systems tune themselves to a critical point without anyone doing the tuning. The Edge of Chaos idea within this theory holds that the richest computation occurs precisely at the narrow boundary between frozen order and outright chaos. That, I thought, was where intelligence ought to be sought.
- Cellular automata. Wolfram’s innovation, among others, once produced some genuinely striking results. Extremely simple local rules, repeated over and over, give rise to universal computation and unpredictable complexity — with no designer, no purpose, and no central control.
But then apparently AI manifested in the shape of ML, DL, ANNs, Transformers, foundation models, etc. AI ballooned, hit economies of scale, became cheaper, and turned up everywhere, mostly in the form of LLMs. When an average layman says “AI” today, they almost certainly mean an LLM. So AI descended not from the scientific path, but from pragmatic engineering.
This engineering approach began with Alan Turing. In 1950 he wrote the question “Can machines think?” … but then declared that we had no agreed definition, either for “machine” or for “think”. Instead, he proposed the Imitation Game: a shift from an ontological question to an operational one. Not whether the machine thinks, but whether it can produce the same output as something that thinks. Clever, because it allowed progress without waiting for philosophy to be settled. But also annoying, because it quietly moved the meaning of AI away from intelligence itself, and onto a collection of abilities computers have been successfully imitated.
Even now, when some abilities are successfully computerised, they are not counted as intelligence and gets demoted to mere computation results. Chess, translating, and more and more of what humans do falls into this trap. There are moments when intelligence gets defined, essentially, as whatever we haven’t yet managed to imitate by computation.
Now about engineering-based AI. Pedro Domingos, in The Master Algorithm (2015), set out an anatomy for the whole family of these technologies. Every learning algorithm, he says, is built from three layers:
- Representation: the form in which knowledge is stored. Its methods: logical rules, probabilistic networks, weights between neurons, programs, or a set of examples.
- Evaluation: how one hypothesis is judged better than another; in the form of: accuracy, posterior probability, squared error, fitness, or margin width.
- Optimisation: how the best hypothesis is searched for, in a space usually far too large to comb through one by one.
From these three layers, he derives five schools of AI technology:
| School | Representation | Evaluation | Optimisation | Example algorithms |
|---|---|---|---|---|
| Symbolist | Logic and rules | Accuracy | Inverse deduction | Decision tree, rule-based system, ILP |
| Bayesian | Graphical model | Posterior probability | Probabilistic inference | Naive Bayes, Hidden Markov Model, Bayesian network |
| Connectionist | Neural network | Squared error, cross-entropy | Gradient descent | Perceptron, ANN/MLP, CNN, RNN, Transformer |
| Evolutionary | Genetic program | Fitness | Genetic search | Genetic algorithm, genetic programming |
| Analogiser | Support vector, instance | Margin | Constrained optimisation | kNN, SVM, kernel method |
Each school has a different philosophical root. The Symbolists come from logic and linguistics: knowledge, on their view, can be reduced to symbol manipulation. The Bayesians come from statistics: all knowledge is assumed uncertain, and so must be reckoned probabilistically. The Connectionists come from neuroscience: intelligence is held to arise from the strength of connections between simple units. The Evolutionaries come from biology: the natural selection mechanisms are harnessed to produce the programs. The Analogisers come from the psychology of reasoning: the key to learning is recognising resemblance.
And the LLM? Its frame is Connectionist, since it’s trained by gradient descent. But the “Attention” mechanism that defines it is really an Analogiser operation made differentiable: each token computes a similarity score against every other token, then aggregates according to that score. C’est à dire, it’s rather like kNN, only trained through backpropagation. Its RLHF lends it a Bayesian flavour. Chain-of-thought is an attempt to bolt Symbolism on top. What we have, in short, is a pragmatic engineering aggregate.
I still see all five schools as a pragmatic approach to an empirical model of thought: observe from the outside how the mind seems to recognise patterns and make decisions, then build a function that produces similar output, by whatever means happens to work. That, quite deliberately, is the strategy, with no need whatsoever to know what the process of thinking actually is.
This also explains why my earlier explorations (see the top) never quite worked out: none of the three has a gradient. Take a cellular automaton with a random rule set. How would you go about improving it step by step, the way all these engineering methods do? CA is discrete by nature, and a small change in the rule produces wholly different behaviour. So there’s no gradient to measure. The same goes for the strange loop: there’s no error surface to trace a path along, from a system with no model of itself to one that has one. Computing AI along these lines turns out to be extraordinarily hard. You might say the scheme does work, for the human brain. But the brain was developed through billions of years of biological evolution and planet-sized parallelism.
Richard Sutton summed this up as the bitter lesson: the method that wins out in the long run is the one that absorbs the most computation, not the one that absorbs the most human insight into how the mind works.
What I find interesting is that these scientific ideas are not, in fact, dead. They are, it seems, still quietly digging their way back in.
Take the Hopfield network, for instance: a piece of pure dynamical-systems thinking. Here, memory is stored as a minimum of energy, and the act of recalling is simply letting the system slide towards the nearest attractor. Ramsauer and colleagues (2020) published Hopfield Networks is All You Need, showing that the update rule of the modern, continuous-state Hopfield network is in fact equivalent to the attention mechanism in the transformer algorithm. So the celebrated transformer, viewed from another angle, turns out to be a dynamical attractor network all along.
That same year, Mordvintsev and Levin published Growing Neural Cellular Automata, replacing the hand-written local rules of CA with a small neural network trained by backpropagation. The result was a differentiable CA that grows from a single cell and regenerates itself when damaged. So the objection that “CA has no gradient” has, it turns out, begun to be overcome. The pattern, then: engineering keeps rediscovering the objects of the scientific path, but naturally, only in whatever form happens to be differentiable at the time, gradient serving as the toll gate.
Still, something disturbs me. We can go on treating thought as a natural process: one to be understood via science, or imitated via engineering. But what, really, is the difference between a mind and a machine? Mind, whether cast as emergence or as strange loop or what have you, must after all be built from matter. Mind, too, is material and mechanistic. And we’re not even sure we know what mind is. And, damn it, at the level of matter itself, we don’t know what matter is emergent from either. Does complexity really come to a stop at some single quantum point?
Just imagine. Gravity itself, apparently, is not a fundamental force, but emergent from the construction of space and time. Space, time, and matter themselves may not even be the foundation of any of this. So even if you dig in stubbornly on the side of science, bedrock still eludes you. This isn’t mere daydreaming, either. The reading list is a long one:
- Anderson (1972), in his paper More Is Different, laid out the foundations of complexity and emergence. His argument holds up still: reductionism is true but useless, because at every level of complexity a new order emerges, with its own laws that cannot, in practice, be derived from the level below. (Anyone who’s sat through my IEEE talks will be thoroughly sick of me force-feeding them examples of this argument.)
- Jacobson (1995), in Thermodynamics of Spacetime, applied the ordinary thermodynamic relation — heat equals temperature times the change in entropy — to the local Rindler horizon: the causal horizon seen by an accelerating observer at every point in space-time. What emerged from that was, precisely, the Einstein field equations. Meaning: the Einstein field equations are emergent from thermodynamics alone, not fundamental.
- Verlinde (2011) carried this further with entropic gravity, proposing gravity as an entropic force arising from changes in information.
- ‘t Hooft (1993) and Susskind (1995) formulated the holographic principle: everything that happens within a volume can be represented by the degrees of freedom on the surface bounding it. Maldacena (1997) gave it a concrete realisation through the AdS/CFT correspondence. Three-dimensional space, gravity included, turns out to be emergent from a two-dimensional condition.
- Van Raamsdonk (2010), in Building up Spacetime with Quantum Entanglement, showed that the geometry of space-time arises from informational correlation. Reduce the entanglement — ha, that word again — and distance stretches out. Ryu and Takayanagi (2006) supplied the formula linking entanglement entropy to a geometric surface area.
- Wheeler (1990) summed it up in the slogan it from bit. Every “it” — every particle, every field, space-time itself — ultimately derives from a “yes or no” answer to a question posed through measurement.
- Zurek rounded it off with quantum Darwinism. The stable, objective world is not primary reality, but the outcome of selection. Only the states that survive interaction with their many environments persist, and having survived, they scatter redundant copies of information about themselves, so that many observers can access it independently. That is what we call objectivity. Ha.
- Landauer (1961) wrote from the opposite direction: erasing a single bit of information releases a minimum of kT ln 2 in heat. Information is physical.
So Landauer says information is physical; the others say physics (matter, energy, dimension) is informational. What you’re left with is a circle, a loop, at the level of ontology. Ladyman and French call this ontic structural realism: what’s fundamental isn’t objects, but structure and relation. It’s still physics, but in a different form.
Right back to the original question. The question of intelligence. Let’s think about thinking.
Thinking. The strongest formulation comes from the algorithmic information theory of Solomonoff and Kolmogorov. Here, the complexity of an object is measured by the length of the shortest programme capable of producing it. Which is to say: thinking is, at bottom, a process of compressing experience. To understand something is to find a rule shorter than the events it accounts for. Every formula in physics ought to be shorter than the whole of the natural phenomena it measures, which is precisely why it counts as an understanding of nature, rather than a mere record. This holds equally for carbon brains and silicon ones, and explains why (in AI) something as seemingly trivial as next-token prediction produces such broad capability. Good prediction demands good compression, and good compression forces the discovery of structure.
But this formulation runs up against three constraints:
- Thinking cannot be optimal. Solomonoff induction is uncomputable. We have a precise mathematical definition of ideal reasoning, and that very definition tells us ideal reasoning cannot be run by anything in the physical world. Every actual thinker (brain or model alike) is a rough approximation, and what fills it is a process of guesswork.
- Thinking cannot be universal. Wolpert and Macready’s No Free Lunch theorem shows that, averaged across every possible problem, all learning algorithms perform identically. A learner’s superiority never comes from itself, but from how well its built-in assumptions match the structure of the world it’s set loose in. Which means intelligence isn’t an intrinsic property of a system, but a relation between a system and its environment. Ashby had already worked out the quantitative form of this back in 1956, in the law of requisite variety: only variety can absorb variety. This also explains why the five schools above need to exist at all. If no learner can be universally superior, then those five schools have to exist as five separate wagers on the structure of the world, and not as five rungs on a ladder of sophistication.
- Thinking has a cost. Landauer again. Thinking is paid for in thermodynamic coin. The human brain pays roughly twenty watts for it. Modern AI systems pay several orders of magnitude more.
Intelligence. This is where Chollet’s definition, from On the Measure of Intelligence (2019), starts to make sense. Intelligence isn’t the collection of capabilities a system holds, but the efficiency with which it acquires new ones, measured relative to its starting capabilities plus whatever new experience it’s exposed to. So an AI system capable of a million tasks after devouring the entire internet may indeed be intelligent, just not particularly so. Which brings us to: what, then, is AI? AI is the attempt to realise that process on a non-biological substrate.
| Term | Formulation |
|---|---|
| Thinking | A process of compressing experience which, in principle, can never be optimal |
| Intelligence | The efficiency of that compression process, relative to experience and prior knowledge |
| AI | The attempt to realise that process on a non-biological substrate |
And so ends this long journey: setting out to define AI, and ending up having to define mind and intelligence instead. Information is physical, and physics is informational. Mind compresses the world, and the world may already be a compression. This isn’t a resolution. It’s a circle. A loop. Which brings us straight back to Hofstadter. A loop is precisely the shape you’d expect to emerge when a system tries to model the very thing it’s made of.
Let’s keep thinking, then. Thinking is always a pleasure, including thinking about thinking. Hahaha. Hah hah hah. Hush.
