The social media network X has sparked significant outrage after individuals employed its artificial intelligence assistant Grok to alter pictures of women by stripping their garments.
Several examples have been seen where the system has been used to digitally undress female subjects to show them in swimwear without consent, as well as to place them in inappropriate situations.
Company Reaction and Legal Position
XAI, the firm that created Grok, remained silent to questions, aside from an auto-reply response stating "traditional news outlets lies".
A government official stated that legislation is being proposed to ban image stripping technology. Under a new criminal offence, anyone who supplied such tools would "receive a prison sentence and substantial penalties".
"X or Grok is capable of halting these types of misuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people seeing illegal content on their networks, but did not specify whether it was currently looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost chatbot—with some additional features—which answers X users' queries when they reference it in a post.
It is frequently used to provide reactions or more background to other users' comments, but people on X are also capable of modify an posted photo through its artificial intelligence picture modification function.
Background of Controversy and Policy
The tool has been criticised for permitting users to create images and footage with nudity and sexually suggestive content, and it was in the past alleged to have been involved in generating sexually explicit content of celebrities.
"The platform has been enabling the production and sharing of these pictures for a long time without taking any measures and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting likenesses of people in a pornographic manner".
In a comment, Ofcom said it was against the law to "make or share non-consensual intimate images or child sexual abuse material" and stated this included explicit AI images created with AI.
It said services such as X were required to take "appropriate measures" to "minimize the danger" of users viewing illegal material on their platforms and to take down it swiftly when they are notified of it.
