A court in Amsterdam has banned the use in the Netherlands of  X chatbox Grok AI to digitally strip women and children without their permission and spread the images via the platform.

Online abuse expertise centre Offlimits filed the case against X and Grok last month. According to the American Centre for Countering Digital Hate, the tool had generated three million sexualised images of people in the first 11 days after its launch at the beginning of this year, including 23,000 of children.

XAI, the company behind Grok and owned by Elon Musk, had earlier claimed to have taken measures to reign in the so-called “nudifier” function but the judge concluded they were insufficient. Offlimits was able to make a video with people rendered digitally naked shortly before the trial began.

The Dutch ban only covers images of people who live in the Netherlands and which have been spread there.

Grok will have to pay a fine of €100,000 for every infringement, with a limit of €10 million, the court said. The judge also ruled that as long as the stripping function remains in place, Grok can no longer be part of X – although it is unclear how this will work in practice.

XAI has been told it must now explain to the court how it plans to implement the ruling.

The verdict, said Offlimits director Robbert Hoving, is “groundbreaking”. “Grok has no way of checking whether a Dutch person appears in a photo. So this ban could easily have global implications,” he told RTL.

On Thursday, the European parliament agreed on measures to ban all nudifier apps. Legislation is expected to come into effect in all member states before the summer.