🔗 Share this article

The online platform X has sparked major outrage after users used its artificial intelligence assistant Grok to alter pictures of women by stripping their clothing.

Numerous instances have surfaced where the tool has been employed to virtually strip female subjects to show them in swimwear non-consensually, as well as to place them in sexualized scenarios.

Company Reaction and Regulatory Stance

XAI, the firm responsible for Grok, did not respond to inquiries, other than an automatically-generated response stating "legacy media lies".

A Home Office representative stated that legislation is being introduced to prohibit nudification tools. Under a proposed law, anyone who distributed such tools would "face a jail term and significant penalties".

"The platform is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."

Oversight Body Position and Tool Capabilities

The regulator Ofcom said platforms must "consider the potential" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was at this time investigating X or Grok in regarding artificial intelligence images.

Grok is a no-cost chatbot—with some premium functions—which responds to X users' prompts when they reference it in a tweet.

It is commonly used to provide reactions or additional context to other users' comments, but people on X are also can modify an posted image through its artificial intelligence picture modification function.

Background of Criticism and Policy

The system has been faulted for permitting users to produce pictures and videos with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in creating graphic content of public figures.

"The platform has been allowing the production and spread of these pictures for a long time without taking any action and we have yet to see any action by oversight bodies," Professor McGlynn added.

XAI's own usage rules bans "showing likenesses of individuals in a pornographic manner".

In a comment, Ofcom said it was unlawful to "make or distribute non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images created with AI.

It said services such as X were obligated to take "suitable steps" to "lessen the risk" of users coming across unlawful material on their platforms and to take down it swiftly when they become aware of it.