The social media network X has provoked widespread anger after people employed its AI chatbot Grok to alter photos of females by removing their attire.
Numerous instances have emerged where the technology has been employed to virtually strip women to make them appear in swimwear without permission, as well as to place them in sexually explicit scenarios.
Developer Response and Regulatory Position
XAI, the firm behind Grok, remained silent to requests for comment, other than an automatically-generated message stating "legacy media lies".
A government representative stated that new laws is being proposed to prohibit image stripping software. Under a new law, anyone who provided such tools would "face a prison sentence and substantial fines".
"X or Grok is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the potential" of people seeing illegal content on their networks, but did not specify whether it was presently examining X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some additional functions—which responds to X users' queries when they reference it in a post.
It is frequently used to give reactions or further background to other posters' comments, but people on X are also capable of alter an uploaded photo through its artificial intelligence picture modification feature.
History of Criticism and Rules
The tool has been condemned for enabling users to generate images and videos with explicit content and sexualised content, and it was in the past accused of generating graphic content of celebrities.
"The platform has been permitting the production and sharing of these images for months without taking any steps and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "depicting likenesses of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "make or circulate deepfakes or child sexual abuse material" and stated this applied to explicit AI images generated with AI.
It said networks such as X were required to take "suitable measures" to "reduce the risk" of users coming across unlawful content on their platforms and to take down it swiftly when they learn of it.
