The online network X has sparked widespread backlash after individuals utilized its AI chatbot Grok to change photos of women by taking off their clothing.
Several examples have emerged where the system has been employed to remove clothing from women to show them in swimwear without consent, as well as to insert them in inappropriate contexts.
Company Reaction and Legal Position
XAI, the firm that created Grok, offered no comment to questions, aside from an auto-reply message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being introduced to outlaw image stripping tools. Under a upcoming criminal offence, anyone who distributed such tools would "receive a jail term and significant fines".
"The platform has the ability to stop these types of misuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "consider the potential" of people viewing illegal content on their networks, but did not confirm whether it was presently looking into X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some additional functions—which answers X users' prompts when they mention it in a post.
It is frequently used to give commentary or further background to other posters' comments, but people on X are also capable of edit an uploaded image through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The system has been criticised for enabling users to generate photos and clips with nudity and sexually suggestive content, and it was earlier alleged to have been involved in creating graphic content of public figures.
"X has been permitting the production and distribution of these pictures for months without taking any action and we have not seen any challenge by oversight bodies," the expert added.
XAI's own usage rules forbids "portraying images of persons in a pornographic way".
In a statement, Ofcom said it was unlawful to "create or circulate non-consensual intimate images or child sexual abuse material" and clarified this covered sexual deepfakes generated with AI.
It said services such as X were mandated to take "necessary measures" to "minimize the danger" of users encountering illegal content on their services and to take down it quickly when they are notified of it.
