A Hidden AI Abuse Apps Economy Inside Mainstream App Stores
Behind polished app store storefronts, a large ecosystem of AI abuse apps is thriving. An investigation by the Tech Transparency Project found 47 “nudify” apps in Apple’s App Store and 55 in Google Play, together downloaded 705 million times. These tools are built to strip clothing from images without consent, directly clashing with both companies’ public bans on non‑consensual sexual content. Yet Apple and Google still take their standard 15–30% commission from in‑app purchases and subscriptions, quietly sharing in a USD 122 million (approx. RM563 million) marketplace for AI‑driven image abuse. The problem is not just passive hosting. Store search systems actively surface these apps, turning non‑consensual deepfake tools into mainstream, one‑tap downloads for millions of users, while official policy pages continue to tout strict rules protecting users’ privacy and digital safety.
How Search, Ads and Design Undermine Content Moderation Claims
The scale of this ecosystem exposes serious gaps in platform accountability and content moderation. The Tech Transparency Project found that simply typing “AI NS” into Apple’s App Store triggers autocomplete suggestions like “image to video ai nsfw,” steering users toward nudify apps. Around 40% of search results for terms such as “undress” return apps capable of creating nude images of women, despite policy language against sexual exploitation. Some apps even carry age ratings suitable for children as young as four, underscoring how safety controls are misaligned with the products’ actual use. Paid advertisements and ranking algorithms further amplify these tools, indicating that enforcement failures are baked into app discovery and monetisation systems. The result is a store environment where non‑consensual AI tools are treated as ordinary entertainment, even as companies promote their commitment to digital safety.
Civil Rights Groups Push for Legal Consequences, Not Promises
Frustrated by years of warnings, 54 civil rights and consumer organizations are now demanding legal action over what they describe as an AI abuse apps marketplace worth USD 122 million (approx. RM563 million). In a letter delivered to the National Association of Attorneys General’s spring conference, advocates argue that appeals to corporate conscience have failed. UltraViolet Action campaign director Jenna Sherman says Apple and Google are “actively directing users to the tune of millions of dollars” toward abuse tools and insists that “the time has passed to appeal to the conscience” of tech CEOs. Ben Winters from the Consumer Federation of America echoes that these companies will not end the abuse “out of the goodness of their own heart.” With new app‑store accountability laws coming online, the coalition wants prosecutors to test whether existing consumer protection and safety statutes can finally force meaningful enforcement.
The Human Cost and the Future of Platform Accountability
Beyond policy violations, the rise of AI nudify apps represents a growing digital safety crisis. Dr. Crystal Cavalier of the Missing Murdered Indigenous Women Coalition warns that AI deepfake abuse is creating “a new frontier of digital violence,” intensifying existing patterns of gendered and racialised harm. She argues that “human dignity, bodily sovereignty, and the safety of future generations must come before corporate profit,” highlighting how monetised image abuse can escalate offline harassment and violence. The continued presence of these apps, despite public bans, highlights a structural disconnect: platform content moderation systems are optimised for growth and revenue, not for preventing non‑consensual exploitation. Whether new legal pressure will compel Apple and Google to align their enforcement with their stated values remains uncertain, but the current gap between promises and practice is eroding trust in the very platforms that claim to keep users safe.