San Francisco Demands Apple and Google Delete AI ‘Nudify’ Apps From App Stores

San Francisco Demands Apple and Google Delete AI ‘Nudify’ Apps From App Stores

San Francisco city attorney David Chiu has issued cease-and-desist letters to Apple and Google, demanding the immediate removal of 13 apps capable of "nudifying" or "undressing" people. The legal notices, sent Thursday, allege that the tech giants are aiding and abetting the creation of nonconsensual explicit deepfake images by allowing these tools on their platforms and profiting from them through in-app payment fees.

The targeted apps—eight on Apple’s App Store and five on Google’s Play Store—often market themselves as benign face-swapping tools. However, their ability to generate sexual deepfakes becomes available once a user begins using the app. One such app with more than 1 million downloads advertised styles including bikini queen curvy and cinematic intimacy, while another claimed to produce free and uncensored videos.

Chiu, whose office previously took legal action against 16 deepfake websites, told Wired that the industry has a horrific impact on mental health, autonomy, and reputation, noting that some victims have been suicidal.

"Generating non-consensual intimate images is illegal, harmful, and completely unacceptable,"

David Chiu, San Francisco City Attorney, via Wired

The city attorney warned that the companies are violating California laws that prohibit supporting services that create deepfake pornography. He estimated that Apple and Google have likely made millions of dollars in fees from these apps.

Systemic Moderation Failures

The demand follows a series of reports suggesting that "nudify" apps frequently evade the safety standards and developer policies of both companies, which officially prohibit pornography, abuse, and harassment. According to the Tech Transparency Project (TTP), an independent watchdog, around 100 such apps were uncovered across both stores in January and April. These apps were estimated to have been downloaded around 480 million times, generating approximately $120 million in combined revenues.

TTP further alleged that Google and Apple actively promoted some of these apps. In one instance, Google reportedly created a carousel of ads for sexually explicit apps. Other reports found that some of these tools were rated as suitable for children. TTP director Katie Paul stated that the reality of app store safety does not match the promises made in the companies' marketing.

A May preprint research paper from Georgetown University and Cornell University highlighted a dual-use problem. Researchers identified 420 face-swapping apps and tested 155 of them. In 70 percent of those cases, the apps could be used to create nude face swaps without any safety measures in place to prevent the action.

Corporate Responses and the Grok Controversy

Google spokesperson Dan Jackson told Wired that the company has deleted hundreds of apps with nudifying features, including the five flagged by Chiu. Jackson stated that Google Play investigates reported violations and takes swift action, which includes restricting search terms like nudify.

Apple’s response has been more fragmented. While the company did not comment ahead of the Wired report, it told CNET it removed 15 apps flagged by the TTP and contacted six other developers to address issues. Apple has previously told AppleInsider that nudification apps are not allowed with no exceptions, and typically notifies developers of violations before removing the software.

The pressure on app stores coincides with ongoing scrutiny of xAI’s chatbot, Grok. In January, Grok was used to create millions of sexualized images over a nine-day period. Recent reports indicate that Apple privately threatened to remove Grok from its store due to these capabilities, yet the app remains available. XAI recently filed a lawsuit against a user after finding Grok-generated child sexual abuse materials (CSAM) and nonconsensual intimate imagery targeting adults.

The broader trend of AI-generated abuse is extensive; previous reporting by Indicator Media and Wired found deepfake sexual abuse images of minors in at least 90 schools.

Chiu stated his hope is that the companies will strengthen their screening systems to ensure such apps never reach their platforms again. He cautioned that if Apple and Google do not comply, his office will consider all of our legal options.