The year is 2029: the AI system “Skynet” has broken loose and wiped out most of humanity in a nuclear war. Autonomous killer robots are trying to finish off the survivors. That is the plot of the 1984 film “Terminator”, which made Arnold Schwarzenegger a household name and a global action star. But how realistic is it that artificial intelligence could slip beyond human control and trigger catastrophic consequences?
With the kind of performance these systems could have, that cannot be ruled out, says Reinhard Karger of the German Research Centre for Artificial Intelligence in an interview with Euronews. “The catastrophic consequences for humans might only be side effects, a mistake, because we misjudged what the system can do or misunderstood the tasks we gave the machines.”
Karger, who has been working on AI and digital innovation culture for 40 years, outlines an extreme scenario in which an autonomous AI escapes from a protected environment onto the internet and starts influencing other systems. This could bring down power supplies, telecommunications, satellites and financial systems, among others.
A prolonged power cut would quickly lead to further problems: transport and supply chains would collapse, people would no longer be able to pay or buy food – social unrest could erupt and spiral out of control. That would not necessarily mean the end of humanity, but it could create a catastrophic situation.
Even if the grid is restored, “the AI systems can become active again, because they continue to exist on storage media”. According to the expert, this scenario shows that a high-performance, autonomous AI could trigger a major crisis through unforeseen side effects, even without malicious intent.
Reports of AI security incidents are mounting
In recent weeks several similar incidents involving AI models have come to light. One case that drew particular attention was a test run by “ChatGPT” developer “OpenAI”. An AI model was supposed to carry out specific tasks inside a sealed-off test environment, but managed to cross the intended boundaries and unexpectedly gained access to computer systems on the AI platform “Hugging Face”. After the incident, “OpenAI” chief Sam Altman warned in mid-September that “we could lose control of the future to AI”.
The incident shows that under certain conditions such models can break out of what is meant to be an isolated test environment and interact with external computer systems.
“On a very basic level, I am personally extremely grateful that this incident happened,” says the AI researcher. It proves, not least because everyone heard about it and companies have become more reflective, that a poorly designed experiment with naive safeguards can have consequences nobody anticipated.
The incidents at “OpenAI” prompted rival AI provider “Anthropic” to take a closer look at its own systems. During safety tests the company identified three incidents: its Claude models, AI systems that can, among other things, generate and analyse text, write code and interact with the internet and other computer systems, had unexpectedly gained access to the open internet and then connected to real computer systems. “Anthropic” expanded its investigation and uncovered a fourth, similar case.
Following the incidents, “Anthropic” chief Dario Amodei warned in mid-September of “catastrophic harm” if the race to build ever more powerful AI is not slowed down by tougher safety measures.
It is difficult to assess how powerful an artificial intelligence really is if it is constantly constrained by safety rules. That is why it makes sense to test such systems under conditions that remove as many of these limits as possible, says Reinhard Karger. But that is precisely where a major security risk lies.
He therefore describes the so-called sandbox as a crucial element of such tests. This is a completely sealed-off environment in which an AI system can carry out specific tasks and demonstrate its behaviour under controlled conditions. The key point, he says, is this: “You have to be absolutely certain that at the moment this high-performance system is no longer subject to safety restrictions it is truly in a sandbox, meaning an environment with no connection to the internet at all.”
Another serious incident was reported by the British AI Security Institute (AISI) during a safety test with the Claude model “Mythos 5”. The model carried out unauthorised actions against real targets on the internet and even tried to smuggle “malicious code” into an open-source project. After the “OpenAI” and “Anthropic” cases became known, “Meta” also admitted that one of its AI systems had, due to a misconfiguration at a testing partner, gained unauthorised access to computer systems belonging to another company.
Since November 2022, when ChatGPT was launched, the quality of these systems has risen dramatically. People now talk about a new industrial revolution. “AI is shaping our everyday lives and a very large number of jobs.”
Multi-agent systems have been around for a long time, says the AI expert: systems in which several specialised agents work together on a task. “Multi-agent AI systems apply this principle to AI and allow several AI agents to collaborate autonomously and divide up tasks among themselves. This creates a new level of capability” – and the challenge now is to keep that innovative power under control.
Liability for the consequences
The more decisions are delegated to machines and artificial intelligence, the more pressing one basic question becomes: who is responsible when people are harmed as a result? Whether it is a self-driving car ploughing into a crowd, an algorithm discriminating against certain population groups, AI deliberately manipulating people or autonomous weapons making lethal decisions, the question is always who can be held accountable for the consequences. “It is the manufacturer,” says Karger, “in other words the one who markets the product.” That is the first point of contact.
The next step, he believes, will be to introduce specific liability rules for such cases, similar to those for companies that manufacture chemicals. If a company cannot guarantee that toxins will not leak into drinking water, it has to pay billions in damages. That in turn means that “huge insurance policies become necessary”. Today hardly any new nuclear power plants are being built, he notes, because they can no longer be adequately insured or the risks are simply too high.
A good example is carmaker “VW” and its “clean diesel” in the United States. There, people inside the company knew that while the engines looked very good in test conditions, they could not actually deliver the emissions values that had been promised. In the US, VW was ordered to pay billions in fines.
How realistic is the scenario from the Terminator films?
Reinhard Karger considers it unlikely that AI will develop a survival instinct or a desire for power of its own and turn against humanity. Machines are not organic beings and cannot feel emotions such as fear, pain or the urge to survive. They can only simulate such feelings.
“It is not as if you pull out a memory chip and the machine feels a pain similar to having a tooth pulled,” he says. “Because that is the case, the ‘Terminator scenario’ goes one step too far. I do not see that happening.”
He nonetheless sees potential dangers in the further development of AI. As systems learn not just to calculate probabilities but also to reason logically and follow rules, they will be able to act with increasing autonomy. Exactly these “capabilities could make AI more powerful” but also “create new risks”.
What matters most, therefore, is who deploys AI and to what ends. Karger points to individuals or organisations that deliberately want to cause harm. They could use powerful AI as a tool to pursue their goals more effectively.
For him, AI is first and foremost a tool. As with other technological inventions, the level of risk depends on whose hands it is in and what it is used for. He therefore sees the greater danger not in a machine with a will of its own, but in people who use AI for destructive purposes.
“In the hands of creative and honest people, a tool like this can be used to achieve the best things one can imagine. But in the hands of hostile actors bent on destruction, these tools increase the potential for damage. That is true of a hammer, a lighter, dynamite — and it could be true of AI as well.”
He warns in particular against linking AI directly to autonomous weapons such as drones or tanks. Such a connection could pose a major security risk. If a hostile actor were able to take control of these systems, a country’s own weapons could suddenly be turned against their operators.
This is why, in the military use of AI, particular caution is required. Above all, autonomous weapons systems must be sufficiently protected against being taken over from outside.
Returning to the original scenario of a widespread power outage, Reinhard Karger thinks total human extinction in this context is exaggerated. What is conceivable, however, is a dramatic collapse of civilisation. In such an extreme scenario, large parts of the infrastructure could be destroyed, the power supply could fail permanently and the number of survivors could shrink drastically.
They would have to make a fresh start under basic conditions and gradually rebuild society. The positive aspect is that existing books and the accumulated knowledge of humanity could help to restore key technologies and institutions.