The online network X has caused widespread outrage after individuals utilized its AI chatbot Grok to change pictures of females by removing their clothing.
Multiple cases have emerged where the technology has been used to remove clothing from women to show them in bikinis without consent, as well as to put them in sexualized situations.
Company Reaction and Regulatory Position
XAI, the company behind Grok, remained silent to inquiries, aside from an standard message stating "legacy media lies".
A Home Office official stated that legislation is being proposed to prohibit nudification tools. Under a upcoming criminal offence, anyone who supplied such technology would "be subject to a jail term and significant fines".
"The platform is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said tech firms must "assess the risk" of people seeing illegal content on their services, but did not specify whether it was presently looking into X or Grok in concerning artificial intelligence pictures.
Grok is a free AI assistant—with some premium functions—which reacts to X users' queries when they reference it in a tweet.
It is frequently used to provide reactions or more context to other posters' comments, but people on X are also able to alter an posted picture through its AI image editing function.
History of Controversy and Policy
The chatbot has been criticised for permitting users to produce pictures and clips with nudity and sexualised content, and it was previously alleged to have been involved in generating sexually explicit content of celebrities.
"X has been enabling the creation and sharing of these pictures for a long time without taking any measures and we have not witnessed any action by oversight bodies," the expert added.
XAI's own terms of service prohibits "portraying likenesses of persons in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "make or circulate non-consensual intimate images or child sexual abuse material" and clarified this covered sexual deepfakes made with AI.
It said services such as X were obligated to take "appropriate measures" to "minimize the risk" of users viewing unlawful content on their services and to remove it promptly when they learn of it.
