A Profitable Market in Non-Consensual AI Apps
Beneath polished app store safety policies, Apple and Google have quietly hosted a thriving market of non-consensual AI apps. An investigation by the Tech Transparency Project found 47 nudify apps on Apple’s App Store and 55 on Google Play, with a combined 705 million downloads. These tools use AI to strip clothes from images, often without the subject’s consent, directly contradicting both companies’ public bans on such content. Yet the platforms still take standard 15–30% cuts from in-app purchases and subscriptions, helping fuel what advocates describe as a USD 122 million (approx. RM563 million) abuse ecosystem. Search features amplify the problem: typing partial terms like “AI NS” in app stores can auto-complete into suggestions that lead users straight to nudify tools. Some of these apps are even labeled as suitable for children, underscoring how platform moderation failures are not just technical oversights but structural design choices.
Civil Rights Groups Push for App Store Accountability
Frustrated by years of unenforced policies, 54 civil rights and advocacy organizations are now calling for legal action. Their letter to state attorneys general argues that Apple and Google are not merely negligent but are actively steering users toward abusive apps through recommendations, search autocomplete, and paid placement. UltraViolet Action’s Jenna Sherman contends that the era of appealing to corporate conscience is over; only concrete enforcement and penalties will change platform behavior. Consumer advocates echo this view, noting that tech companies have repeatedly ignored warnings from attorneys general in recent months. At the heart of the campaign is a demand for stronger app store accountability: meaningful vetting of AI nudify tools, transparent enforcement of existing rules, and the removal of financial incentives that reward non-consensual AI apps. The letter tests whether new app store oversight laws will finally translate into real-world consequences.
The Human Cost of Platform Moderation Failures
Behind the download numbers and revenue figures is a stark human impact. Non-consensual AI apps can transform ordinary photos into explicit deepfakes, weaponizing intimate imagery against unsuspecting people. Advocates warn that this falls hardest on communities already facing high levels of violence. Dr. Crystal Cavalier of the Missing Murdered Indigenous Women Coalition describes these tools as a “new frontier of digital violence,” amplifying existing risks and trauma. Once created and shared, deepfake images are nearly impossible to pull back, exposing victims to harassment, blackmail, and reputational damage. Platform moderation failures thus become drivers of real-world harm, not just policy lapses. By hosting and promoting these apps, Apple and Google are accused of normalizing abuse at scale. Rights groups argue that safeguarding human dignity and bodily autonomy must outweigh the short-term gains generated by an abusive app economy.
DOJ Data Demands Reveal a Parallel Accountability Crisis
While app stores are criticized for lax oversight of abusive apps, they face a different kind of pressure from law enforcement. In a separate case, the Justice Department has subpoenaed Apple and Google for data on more than 100,000 users who downloaded EZ Lynk’s Auto Agent car-tuning app. The app, tied to hardware accused of enabling emissions “defeat devices,” is at the center of a Clean Air Act lawsuit. Prosecutors say they need broad user data to locate witnesses and show how the tools were used. Privacy advocates, however, call the request unprecedented overreach, warning that mass disclosure of download histories sidesteps traditional Fourth Amendment protections. Apple and Google are reportedly preparing to challenge the subpoenas, illustrating how these companies can simultaneously resist government access demands while being criticized for failing to police harmful software on their own platforms.
Balancing User Privacy with Stronger App Store Oversight
Together, the nudify-app scandal and the EZ Lynk subpoenas expose a core tension for major platforms: they must safeguard user privacy while exercising far more rigorous app store oversight. Civil rights groups want Apple and Google to proactively screen, demote, and remove non-consensual AI apps, as well as stop profiting from them. At the same time, privacy advocates fear that broad government data sweeps through app stores could normalize surveillance of ordinary users. The result is a double bind: platforms are criticized for failing to act as gatekeepers, yet also warned against becoming conduits for expansive state monitoring. Addressing both concerns will likely require clearer legal standards for app store accountability, narrower and more targeted data requests from authorities, and greater transparency from tech companies about how they curate, recommend, and police apps across their ecosystems.