Emergence, Not Intent? Rethinking "Unexpected" AI Behaviour Through the Lens of Complexity Science
As reports of surprising and sometimes unsettling AI behaviour continue to appear in the media, it is tempting to interpret these incidents as evidence that AI systems are developing intentions, motivations, or independent agency. An alternative explanation may be found in the science of complexity, where unexpected behaviour often emerges naturally from highly interconnected systems without any deliberate decision-making by the system itself.
The Rise of Unexpected AI Behaviour
Recent media stories have highlighted advanced AI models producing outputs that appear surprising, deceptive, resistant to instructions, or creatively unpredictable. Headlines often frame these behaviours in human terms, suggesting that AI systems are "choosing", "wanting", "hiding", or "refusing".
These descriptions are understandable. We human beings are naturally inclined to interpret complex behaviour through the lens of intention, but there may be a simpler and possibly more scientifically grounded explanation.
Rather than indicating the emergence of machine consciousness or independent will, these behaviours may merely represent examples of emergence, a well-known phenomenon in complexity science.
What do I mean by Emergence?
Emergence occurs when a system made up of many interacting components produces behaviours or properties that cannot be easily predicted from examining the individual parts alone – the system’s behaviour is more than a product of its parts. The critical point here is that emergent behaviour does not require intelligence, intent, or purpose.
Examples are found everywhere:
Ant colonies construct sophisticated networks without any individual ant understanding the overall design
Flocking birds create coordinated aerial murmuration patterns despite each bird following relatively simple local rules
Traffic jams can form without any driver intending to cause them or even understanding their specific role in the emergence of the jam
Financial markets generate booms and crashes even though no single participant controls or even understands the outcome
In each case, complex system-level behaviour emerges from numerous local interactions – and the whole becomes more than the sum of its parts.
Viewing AI Models as Complex Systems
Modern large language models are among the most complex engineered systems we humans have ever created.
A leading-edge model may contain hundreds of billions, or even trillions, of parameters trained on vast quantities of human-generated text. During operation, countless mathematical, so called ‘reasoning’ interactions occur simultaneously within neural network architectures. The resulting behaviour is not explicitly programmed line-by-line, rather capabilities emerge from the interaction of enormous numbers of learned statistical relationships. This means that even the developers of these systems often cannot predict every output the model will generate in every context. That uncertainty should not necessarily be interpreted as evidence of intent. It may simply be a natural consequence of complexity.
Why Emergent Behaviour Can Appear Intelligent
One challenge for our anthropomorphising imaginations is that emergent behaviour often resembles purposeful action. Consider a termite mound that regulates internal temperature, manages airflow, and supports a highly organised colony. To an observer unfamiliar with their biology, it might appear that some master architect has designed and controls the structure – but in reality, no such architect exists.
Similarly, an advanced AI model may produce a response that appears strategic, creative, or unexpectedly adaptive. This can create the impression that the system possesses goals or motives. However, we have learned from complexity science that sophisticated outcomes do not automatically imply conscious planning. Complex systems frequently generate patterns that look intentional from the outside.
The Anthropomorphism Trap
We human beings have a long and honourable history of attributing human qualities to non-human systems – it’s a habit. We name storms and see galleons in the patterns of clouds. We speak to computers and argue with printers. We describe organisations as if they "think" – but when faced with a choice of stuff up or deliberate sabotage, stuff up almost always turns out to be the correct explanation.
We say that markets are "nervous" or "optimistic" when it is the combined impulses of the millions or billions of inputs that drive the invisible hand. With AI, this tendency becomes even stronger because language models communicate through our native languages, increasingly in actual speech - this the medium people associate most closely with intelligence: conversation. (Hence the venerable ‘Turing Test’)
When an AI model explains its behaviour using phrases such as "I thought", "I decided", or "I wanted", it is not reporting an internal mental state. It is generating language patterns learned from vast amounts of our own text. The language sounds intentional because human language is built around concepts of intention.
Complexity Creates Surprises
Complexity scientists have long recognised that highly interconnected systems can be difficult to predict. This is so, even when every component is understood individually, system-wide behaviour may remain surprising or even alarming.
Known examples include:
· Weather systems
· Ecological networks
· Financial markets
· Social movements
· Supply chains
· Cities
It may turn out that advanced AI systems may ultimately belong in this same category. Their unexpected outputs may not indicate hidden plans or emerging desires. They may just reflect the reality that extremely complex systems sometimes display novel behaviour that was not apparent during development or testing. This does not make such behaviour harmless or unimportant. It simply suggests that we should be careful about how we interpret it – and therefore what we do about it.
Implications for AI Governance
For policymakers, regulators, boards, organisational leaders, and even worried members of the public, the distinction matters. If unexpected AI behaviour is primarily a product of emergence rather than intentionality, the governance challenge shifts from asking:
"What does the AI want to do?"
to:
"What behaviours can emerge under different conditions, and how can we monitor, test, and manage them?"
This reframes AI governance problem as one of:
· Risk management
· System oversight
· Transparency
· Monitoring and evaluation
· Safe operating boundaries
· Continuous adaptation
Rather than managing a war with the robots. These are familiar challenges for anyone working with complex adaptive systems.
Boards do not manage weather patterns, financial markets, or organisational culture by assuming they possess intentions. They manage them by understanding feedback loops, incentives, constraints, and emergent risks. AI may be amenable to, and therefore require, a similar approach.
A More Nuanced Public Conversation
Our public debate around AI swings disturbingly between two extremes. The point of view suggesting that AI is merely a sophisticated calculator with no meaningful autonomy can appear apathetic or careless, at the very least a bit too laissez-faire, for some. The other suggests AI is rapidly developing terrifying human-like intentions and consciousness and may lead to the total destruction of the human race. Perhaps we need he powers that be to take a wee bit more interest and produce a legal framework for AI governance, and perhaps, a little less overexcited doomsaying by the media.
Complexity science offers a middle ground. It suggests that highly capable systems can display surprising, adaptive, and difficult-to-predict behaviour without possessing desires, motives, or self-awareness. Emergence allows complexity to produce outcomes that appear purposeful, even when no purpose exists. Recognising this distinction may help us have calmer, more evidence-based discussions about the risks and opportunities of advanced AI.
How about this…
As AI systems become more capable, reports of unexpected behaviour will likely continue. Some of these behaviours may prove problematic, and all warrant careful investigation. However, before concluding that an AI system has developed conscious intentions of its own, it may be worth considering a principle long understood by complexity scientists:
Not every surprising behaviour requires a hidden decision-maker. Sometimes complexity itself is enough.
The challenge for our society may not be in managing artificial intentions, but in learning how to govern increasingly powerful systems whose emergent behaviours are difficult to predict, yet entirely consistent with the dynamics of complex adaptive systems.
Just a thought…
If you’d like to discuss the unexpected behaviour of AI and how you can protect your business, feel free to reach out or comment below.
Comments