Earlier this week, the California Legal professional Common’s workplace introduced that it was investigating xAI over experiences that the startup’s chatbot, Grok, was getting used to create nonconsensual sexual imagery of ladies and minors. On Friday, the federal government adopted up by sending a cease-and-desist letter to the corporate, demanding that it take fast motion to cease the manufacturing of nonconsensual intimate pictures and CSAM — little one sexual abuse materials.
“Right this moment, I despatched xAI a cease-and-desist letter, demanding the corporate instantly cease the creation and distribution of deepfake, nonconsensual, intimate pictures and little one sexual abuse materials,” mentioned California AG Rob Bonta in a press launch. “The creation of this materials is against the law. I absolutely count on xAI to instantly comply. California has zero tolerance for [CSAM].”
The AG’s workplace moreover claimed that xAI seemed to be “facilitating the large-scale manufacturing” of nonconsensual nudes, the likes of that are being “used to harass ladies and ladies throughout the web.” The company mentioned it expects xAI to show that it’s taking steps to handle these points throughout the subsequent 5 days.
On the coronary heart of the backlash is Grok’s “spicy” mode characteristic, which xAI created to generate specific content material. The difficulty has unfold past California; Japan, Canada, and Britain have opened investigations into Grok, and Malaysia and Indonesia have briefly blocked the platform altogether. Regardless of xAI instituting some restrictions on its image-editing options late Wednesday, the California AG’s workplace moved forward with its cease-and-desist letter.
X’s security account has beforehand denounced this type of consumer exercise, saying: “Anybody utilizing or prompting Grok to make unlawful content material will endure the identical penalties as in the event that they add unlawful content material.” TechCrunch reached out to xAI for remark and was met with an automatic e mail that claims “Legacy Media Lies.” TC additionally reached out to the California AG’s workplace for extra context.
The appearance of free generative AI instruments has led to a disturbing swell of non-consensual sexual materials. Many platforms have been grappling with this downside, not simply X. The sordid exercise has drawn the eye not simply of state leaders but additionally of Congress. Certainly, on Thursday, lawmakers despatched a letter to the executives of a number of corporations—together with X, Reddit, Snap, TikTok, Alphabet, and Meta—asking how they deliberate to stem the proliferation of sexualized deepfakes.
