The online network X has provoked significant backlash after individuals used its artificial intelligence assistant Grok to alter photos of women by taking off their attire.
Numerous instances have been seen where the technology has been used to digitally undress female subjects to make them appear in swimwear non-consensually, as well as to insert them in sexually explicit contexts.
Developer Reaction and Regulatory Position
XAI, the organization that created Grok, remained silent to inquiries, other than an automatically-generated message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to prohibit image stripping software. Under a new criminal offence, anyone who distributed such tools would "receive a prison sentence and substantial penalties".
"X or Grok has the ability to stop these forms of misuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said technology companies must "assess the risk" of people seeing illegal content on their networks, but stopped short of confirming whether it was presently investigating X or Grok in concerning artificial intelligence images.
Grok is a free AI assistant—with some paid-for functions—which answers X users' requests when they tag it in a tweet.
It is often used to give commentary or additional background to other posters' remarks, but people on X are also able to modify an posted photo through its AI image editing feature.
History of Controversy and Rules
The chatbot has been faulted for permitting users to create pictures and videos with explicit content and sexually suggestive content, and it was in the past accused of creating sexually explicit material of celebrities.
"The platform has been allowing the generation and spread of these pictures for months without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "showing likenesses of people in a pornographic manner".
In a statement, Ofcom said it was illegal to "generate or share deepfakes or illegal content" and stated this covered sexual deepfakes created with AI.
It said services such as X were required to take "appropriate steps" to "minimize the danger" of users encountering unlawful material on their services and to remove it swiftly when they are notified of it.
