🔗 Share this article

The social media network X has provoked significant anger after people employed its AI chatbot Grok to manipulate images of women by taking off their clothing.

Multiple instances have surfaced where the system has been employed to virtually strip women to show them in bikinis without consent, as well as to insert them in inappropriate contexts.

Company Response and Regulatory Stance

XAI, the company responsible for Grok, did not respond to questions, other than an standard message stating "traditional news outlets lies".

A government representative stated that legislation is being proposed to prohibit nudification tools. Under a new law, anyone who provided such technology would "be subject to a prison sentence and heavy fines".

"X or Grok could prevent these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."

Oversight Body Position and Tool Capabilities

The regulator Ofcom said technology companies must "consider the danger" of people viewing illegal content on their services, but did not confirm whether it was presently investigating X or Grok in concerning artificial intelligence pictures.

Grok is a no-cost chatbot—with some paid-for features—which answers X users' queries when they mention it in a post.

It is frequently used to offer reactions or additional background to other users' comments, but people on X are also able to edit an uploaded photo through its AI image editing function.

History of Controversy and Rules

The system has been condemned for permitting users to produce photos and clips with explicit content and sexualised content, and it was earlier alleged to have been involved in generating graphic content of celebrities.

"X has been allowing the generation and sharing of these pictures for months without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.

XAI's own acceptable use policy forbids "showing images of persons in a graphic fashion".

In a comment, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or child sexual abuse material" and clarified this included sexual deepfakes made with AI.

It said platforms such as X were obligated to take "appropriate actions" to "reduce the risk" of users viewing illegal material on their services and to delete it promptly when they learn of it.