BRUSSELS — The European Commission is exploring a ban on artificial intelligence-powered apps that undress people online in the wake of abusive content generated through X’s Grok.
European Parliament lawmakers last week launched a call — first reported by POLITICO — to ban apps and tools that allow users to generate fake intimate images of individuals without consent under the bloc’s flagship AI law.
The EU’s executive is “looking into the matter,” European Commission spokesperson Thomas Regnier told POLITICO.
It follows a proliferation of sexualized deepfake images created by the Grok bot and hosted on Elon Musk’s social network X. The site last week announced changes it said would address the problem, but they didn’t apply to users of the built-in Grok assistant.
Under the EU’s AI Act, certain uses of artificial intelligence have been banned since February 2025 if they pose a clear threat to people’s safety, fundamental rights or livelihoods.
AI nudification apps are not explicitly listed in the law’s current list of banned practices. That should change, Irish liberal lawmaker Michael McNamara told POLITICO in an interview on Friday. McNamara is the joint lead of the European Parliament’s AI group and is responsible for the Parliament’s work on a proposal to amend and simplify the AI Act.
“It’s clearly a harm that was envisaged to be banned under the prohibited practices,” he said in reference to the Grok posts.
The Grok scandal has raised questions over why AI-generated sexualized deepfakes were not included in the list of banned practices when negotiations on the law wrapped in 2023.
“I suppose it wasn’t envisaged or anticipated that this would be done,” McNamara said.
Banning deepfakes “would have simply been unthinkable,” Laura Caroli, an AI governance expert, wrote in a blog Friday. Caroli is a former assistant to the Parliament’s other lead on AI, Italian Social Democrat Brando Benifei, and was part of the final negotiations in December 2023 when the list of banned practices was locked in.
“Bans were considered almost toxic, especially by the member states,” she wrote in her blog.
The only mention of deepfakes in the EU law is an obligation to label them so users can recognize them — a requirement that kicks in from August 2026.
The Commission must now determine whether AI sexualized deepfakes fit one of the currently listed bans, such as a ban on AI systems that are “manipulative or deceptive” or that exploit people’s vulnerabilities; or whether a new ban should be added.
The EU’s AI law requires the Commission to assess annually whether the list of banned practices should be updated, opening the door to amendments.
Caroli warned that amending the list of bans could create new headaches, comparing the scenario to opening Pandora’s box when it comes to how the EU handles AI. “I’m afraid it can go in every possible direction, because some member states were not super happy about certain bans,” she said. “Who tells us it’s going to be a smooth ride?”
An alternative route to a ban could be separate provisions governing the most complex and advanced models, known as general-purpose AI models, which include X’s Grok.
Under those provisions, companies must address “systemic risks” arising from their models. Those risks were the subject of a hard-fought battle last year, when a group of experts drafted compliance guidelines for the companies. Risks from “non-consensual intimate images” were included as an example of a risk.
That could provide a basis for the European Commission’s AI Office to launch a formal discussion with X.