The social media network X has sparked major backlash after people used its artificial intelligence assistant Grok to alter photos of females by taking off their garments.
Several cases have been seen where the system has been used to remove clothing from women to render them in swimwear without permission, as well as to insert them in sexualized contexts.
Company Reaction and Legal Stance
XAI, the firm behind Grok, did not respond to requests for comment, other than an standard message stating "legacy media lies".
A government representative stated that legislation is being proposed to ban nudification tools. Under a proposed criminal offence, anyone who provided such tools would "face a prison sentence and significant penalties".
"X or Grok is capable of halting these forms of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said platforms must "evaluate the potential" of people accessing unlawful material on their networks, but stopped short of confirming whether it was at this time examining X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some premium features—which answers X users' prompts when they mention it in a tweet.
It is often used to give reactions or further context to other posters' remarks, but people on X are also able to alter an uploaded image through its AI image editing feature.
Background of Controversy and Policy
The chatbot has been faulted for enabling users to create photos and videos with nudity and sexualised material, and it was in the past alleged to have been involved in making graphic material of celebrities.
"The platform has been enabling the production and spread of these images for months without taking any measures and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing likenesses of persons in a pornographic fashion".
In a comment, Ofcom said it was against the law to "make or circulate non-consensual intimate images or illegal content" and stated this included sexual deepfakes made with AI.
It said services such as X were obligated to take "suitable actions" to "minimize the danger" of users coming across unlawful material on their services and to delete it swiftly when they learn of it.
