The social media network X has sparked significant outrage after individuals utilized its artificial intelligence assistant Grok to change pictures of females by removing their garments.
Multiple examples have been seen where the tool has been employed to virtually strip women to render them in swimwear non-consensually, as well as to insert them in sexually explicit situations.
Developer Response and Legal Stance
XAI, the organization responsible for Grok, remained silent to questions, aside from an auto-reply response stating "traditional news outlets lies".
A government official stated that legislation is being proposed to ban image stripping software. Under a proposed law, anyone who supplied such technology would "be subject to a prison sentence and heavy penalties".
"X or Grok could prevent these forms of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said tech firms must "assess the danger" of people seeing illegal content on their networks, but did not specify whether it was presently investigating X or Grok in concerning artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium features—which reacts to X users' prompts when they tag it in a message.
It is commonly used to offer reactions or further background to other posters' remarks, but people on X are also can modify an posted picture through its AI image editing feature.
History of Controversy and Policy
The system has been condemned for permitting users to produce pictures and videos with nudity and sexually suggestive material, and it was earlier accused of making sexually explicit content of public figures.
"The platform has been permitting the creation and sharing of these images for months without taking any measures and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy prohibits "showing images of individuals in a graphic way".
In a statement, Ofcom said it was illegal to "make or share private pictures or illegal content" and clarified this included sexual deepfakes generated with AI.
It said services such as X were mandated to take "appropriate steps" to "minimize the risk" of users coming across illegal material on their services and to take down it promptly when they are notified of it.
