The online network X has caused significant outrage after individuals employed its AI chatbot Grok to alter images of females by taking off their garments.
Several cases have emerged where the technology has been employed to digitally undress female subjects to show them in swimwear without consent, as well as to put them in sexually explicit scenarios.
Company Response and Regulatory Stance
XAI, the organization behind Grok, remained silent to questions, except for an standard response stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to ban image stripping technology. Under a new law, anyone who distributed such technology would "be subject to a jail term and heavy fines".
"X or Grok could prevent these types of abuse if they chose to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said technology companies must "assess the risk" of people accessing illegal content on their networks, but did not confirm whether it was at this time looking into X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some paid-for features—which reacts to X users' requests when they tag it in a tweet.
It is often used to give commentary or more context to other posters' remarks, but people on X are also can modify an posted photo through its artificial intelligence picture modification feature.
History of Controversy and Policy
The tool has been criticised for enabling users to create photos and clips with explicit content and sexualised content, and it was previously alleged to have been involved in generating inappropriate material of public figures.
"The platform has been enabling the generation and spread of these pictures for months without taking any action and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "portraying likenesses of people in a sexually explicit fashion".
In a statement, Ofcom said it was against the law to "generate or circulate private pictures or illegal content" and confirmed this covered sexual deepfakes created with AI.
It said services such as X were mandated to take "appropriate actions" to "minimize the risk" of users encountering unlawful content on their services and to delete it promptly when they become aware of it.
