The social media network X has provoked widespread outrage after users employed its artificial intelligence assistant Grok to manipulate images of women by taking off their garments.
Several instances have surfaced where the system has been used to virtually strip women to make them appear in bikinis without consent, as well as to insert them in sexually explicit contexts.
Company Reaction and Legal Stance
XAI, the firm responsible for Grok, remained silent to requests for comment, other than an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being introduced to ban image stripping tools. Under a new criminal offence, anyone who supplied such tools would "be subject to a prison sentence and significant penalties".
"X or Grok is capable of halting these types of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "assess the risk" of people seeing unlawful material on their services, but did not specify whether it was currently investigating X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some paid-for functions—which answers X users' queries when they reference it in a tweet.
It is often used to provide reactions or more context to other users' comments, but people on X are also can alter an posted photo through its AI image editing tool.
History of Controversy and Rules
The chatbot has been condemned for allowing users to create images and videos with nudity and sexually suggestive material, and it was in the past accused of making sexually explicit content of public figures.
"The platform has been permitting the generation and spread of these images for months without taking any action and we have not witnessed any action by oversight bodies," the expert added.
XAI's own terms of service bans "showing images of individuals in a graphic manner".
In a comment, Ofcom said it was against the law to "generate or distribute private pictures or child sexual abuse material" and stated this applied to explicit AI images made with AI.
It said services such as X were required to take "necessary actions" to "reduce the risk" of users coming across unlawful content on their platforms and to delete it promptly when they are notified of it.
