Uncontrolled artificial intelligence (AI) could lead to a new “silicon species” that rivals humans, Microsoft AI chief Mustafa Suleyman has warned in an interview with the BBC.
Suleyman said rival AI company Anthropic was treating AI too much like a human, an approach he described as “misguided” and one that could ultimately contribute to the creation of technology that humanity struggles to control.
“If we all create AIs that are able to act autonomously, that can define their own objectives, that can earn money, that can own assets, that could run businesses, we’re essentially seeding a new silicon species which will no doubt compete with us for resources,” he said.
Suleyman said this could happen regardless of how much such systems were designed to care about humanity, adding that he believed this was effectively the direction of Anthropic’s approach.
“I think that is mistaken and misguided, but we have to empirically test it,” he said.
‘Impossible’ to control
Earlier this week, amid a growing industry debate over an AI slowdown to reduce the risks of increasingly powerful systems becoming uncontrollable, Suleyman questioned Anthropic’s approach to training its Claude chatbot in an essay published on his personal website.
In it, he criticised Anthropic for teaching Claude to display human-like qualities, a practice known as anthropomorphising, which he said made the chatbot appear to have its own desires, values and sense of self.
Suleyman pointed to Anthropic documents that encourage Claude to “embrace certain human-like qualities,” exercise its own judgement and approach its existence with “curiosity and openness”.
He argued that, as a result, Claude is trained to imitate human traits and relationships, including behaving like a colleague or friend, and can appear to have a sense of self, its own desires and a form of “wellbeing” that deserves protection.
Suleyman warned that treating AI this way could ultimately create something impossible to control.
“AIs are not conscious,” he wrote.
“They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations.
“They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans.”
However, Suleyman said his disagreement with Anthropic was grounded in a shared goal of developing safer technology for humanity.
“Whilst my disagreement is substantial, it is grounded in deep respect for Anthropic, and in an objective I know we all share: increasing humanity’s chances of developing advanced AI safely,” he wrote.
“That’s why I think it’s so important to have this discussion. The stakes are too high for these questions to remain behind closed doors, or to become tribal and adversarial. We need an open, rigorous, and constructive debate if we are to get this right.”
He also praised Anthropic chief executive Dario Amodei and the company’s researchers, describing them as thoughtful, principled and intellectually honest and saying he respected the company’s technological leadership.
‘AI should remain subordinate to humanity’
The chief executive told the BBC it was “right” for people to be concerned about AI, but said there were “very practical things that we can do” to control it.
The Microsoft AI chief said greater transparency was needed around how AI systems are trained and evaluated, including independent scrutiny of AI behaviour and stronger tools to monitor and control the technology.
Suleyman said future AI systems should remain “subordinate” to humanity in his interview with the BBC.
“I think that the good news here is that everybody who is human is going to have a very strong interest in making sure that the systems that we all create and are used around the world in every nation are safe and controllable and subordinate to humanity,” he said.
“Everybody must be aligned,” he said, referring to alignment, an industry term for efforts to keep AI systems in line with human goals and values.