A tech journalist and startup advisor with over a decade of experience covering Silicon Valley innovations.
Elon Musk's social media network X has provoked significant backlash after individuals utilized its AI chatbot Grok to change pictures of females by removing their clothing.
Numerous cases have been seen where the tool has been used to remove clothing from female subjects to make them appear in swimwear without permission, as well as to place them in inappropriate situations.
XAI, the organization that created Grok, did not respond to questions, except for an standard message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to prohibit nudification technology. Under a upcoming law, anyone who distributed such technology would "receive a prison sentence and significant penalties".
"X or Grok has the ability to stop these types of abuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
The regulator Ofcom said technology companies must "evaluate the risk" of people viewing unlawful material on their services, but did not confirm whether it was at this time investigating X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some additional functions—which responds to X users' queries when they reference it in a tweet.
It is often used to offer commentary or additional context to other users' comments, but people on X are also can edit an posted image through its AI image editing function.
The chatbot has been faulted for permitting users to create images and videos with explicit content and sexualised material, and it was earlier accused of making sexually explicit material of celebrities.
"X has been allowing the creation and sharing of these images for months without taking any steps and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy prohibits "showing likenesses of people in a pornographic manner".
In a comment, Ofcom said it was against the law to "create or share deepfakes or illegal content" and confirmed this included sexual deepfakes made with AI.
It said platforms such as X were required to take "appropriate steps" to "minimize the risk" of users coming across unlawful content on their services and to remove it quickly when they learn of it.
A tech journalist and startup advisor with over a decade of experience covering Silicon Valley innovations.