The social media network X has provoked significant backlash after people employed its AI chatbot Grok to manipulate images of women by taking off their attire.
Multiple examples have been seen where the tool has been used to remove clothing from female subjects to make them appear in bikinis without permission, as well as to place them in sexually explicit situations.
Company Response and Regulatory Stance
XAI, the organization behind Grok, remained silent to questions, other than an standard message stating "legacy media lies".
A government official stated that legislation is being brought forward to ban nudification tools. Under a proposed law, anyone who provided such technology would "receive a jail term and substantial fines".
"The platform is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said technology companies must "assess the potential" of people seeing illegal content on their networks, but did not specify whether it was currently investigating X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for features—which answers X users' prompts when they reference it in a tweet.
It is commonly used to offer commentary or additional context to other users' remarks, but people on X are also able to alter an uploaded photo through its artificial intelligence picture modification function.
History of Criticism and Policy
The tool has been faulted for allowing users to create pictures and clips with nudity and sexualised content, and it was earlier alleged to have been involved in generating sexually explicit material of public figures.
"X has been permitting the generation and distribution of these pictures for a long time without taking any action and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "showing likenesses of people in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "generate or distribute non-consensual intimate images or child sexual abuse material" and stated this applied to explicit AI images made with AI.
It said services such as X were mandated to take "appropriate measures" to "lessen the danger" of users viewing illegal content on their platforms and to remove it swiftly when they become aware of it.
