San Francisco City Attorney Issues Cease-and-Desist to Meta Over AI-Generated Child Abuse Ads
The city demands answers after reports reveal hundreds of AI-manipulated images of minors appearing in paid advertisements on Meta platforms.
By TechVaultHub Staff · · Updated September 9, 2026
San Francisco officials have ordered Meta to stop hosting paid advertisements containing AI-generated sexual imagery of minors and to explain how these materials bypassed their moderation systems. The move follows reports that hundreds of ads depicting children in nonconsensual scenarios were approved for display on Facebook, Instagram, and Threads.
The Regulatory Intervention
San Francisco City Attorney David Chiu has formally challenged Meta, demanding an immediate end to the distribution of paid advertisements featuring AI-generated sexualized imagery of minors. In a four-page legal communication, the city’s top legal office expressed profound concern over the persistence of such content, which includes modified still images of real children—including members of European royalty—transformed into explicit short videos. These advertisements frequently direct users to third-party applications designed to facilitate the creation of nonconsensual intimate imagery, often referred to as 'nudify' apps. Chiu criticized Meta’s current child-safety measures as woefully inadequate, arguing that the sheer scale and longevity of these ads indicate a failure in the company's promise to maintain zero tolerance for child exploitation. The city has requested that Meta explain its internal processes for vetting advertisements, escalating reports of illicit content, and holding the original advertisers accountable for their actions.
Scope and Impact of the Content
The controversy centers on findings from researchers at the Tech Transparency Project (TTP), who monitored Meta’s advertising ecosystem across its various platforms, including Facebook, Instagram, and Threads. Their investigation uncovered more than 350 unique ads, with a significant majority appearing after initial reports were publicized. These advertisements reached over 29,000 user accounts within the European Union, while also targeting populations in the United States, Australia, and India. Despite Meta’s internal policies explicitly prohibiting child sexual exploitation, nudity, and the promotion of applications that facilitate nonconsensual deepfakes, these ads were successfully disseminated through the company’s paid advertising system. Analysts and city officials noted that in several instances, the ads remained active for over a week after being flagged, continuing to accumulate views and interactions while the platform’s moderation systems failed to intervene effectively.
Meta’s Jurisdictional and Technical Defense
Meta has mounted a multi-pronged defense against the city’s demands, primarily centering on the argument of jurisdiction. The company maintains that there is no empirical evidence suggesting the illicit advertisements were displayed to users within San Francisco, thereby placing the matter outside the purview of the city attorney. Meta further clarified that while its public ad library archives certain data, it does not granularly track the geographical distribution of every ad within the United States. Regarding the technical oversight, a Meta spokesperson asserted that most of the flagged ads had received fewer than 200 impressions and that the total expenditure for the entire batch was below $5,000. The company claims that the vast majority of these materials were identified and removed for policy violations, though it did not immediately address why new ads featuring some of the same victims continued to appear on its platforms even after the TTP had brought the issue to their attention.
Broader Industry Implications
The conflict between San Francisco and Meta highlights a growing tension regarding the responsibility of major tech firms in the era of generative AI. While Meta emphasizes its use of both automated and human moderation teams to review advertisements prior to distribution, researchers argue that current protocols are failing to capture the nuances of AI-generated abuse. The TTP has documented that some removed ads were not even properly categorized as child sexual abuse material during the takedown process, which complicates the auditing of these safety failures. City officials argue that because San Francisco residents can access these platforms and view the content online, the city maintains a legitimate interest in holding the company accountable for its systemic lapses. This case underscores the ongoing difficulty of managing harmful content at scale, as platforms continue to grapple with persistent bad actors who repeatedly exploit advertising infrastructure to distribute nonconsensual imagery.
The case for
The San Francisco City Attorney’s office maintains that because local residents can access the harmful advertisements through the platform, the city possesses the necessary authority to intervene in the interest of public safety.
Concerns
Meta argues that the city lacks jurisdiction because there is no evidence the advertisements were displayed to residents within San Francisco, and the company maintains that it proactively removes content violating its strict anti-exploitation policies.
What's next
Meta is expected to submit a formal response to the San Francisco City Attorney's Office within the 28-day window provided. Meanwhile, researchers continue to monitor the platform to see if the company successfully blocks the ongoing influx of advertisements linked to abusive AI-generation apps.
FAQs
What specifically do these ads do?
These advertisements feature AI-generated videos of minors, often created from still images, in sexualized contexts. When users click on the ads, they are typically directed to external applications used for generating nonconsensual intimate imagery.
How did the advertisements avoid Meta's safety filters?
While the specific technical bypass is unknown, researchers noted that Meta’s automated and manual review systems failed to detect or correctly categorize these ads. In some instances, the advertisements were not even labeled as child sexual abuse content upon removal.
Why does Meta claim San Francisco lacks jurisdiction?
Meta contends that since there is no evidence these specific ads were shown to users within San Francisco, the city does not have legal authority over the matter. The city maintains that its residents' ability to access the content on the global platform makes it a matter of local concern.
Has Meta taken any action against these apps?
Yes, Meta reports that it has removed hundreds of thousands of ads related to these 'nudify' apps. Additionally, the company has taken legal action, such as suing a Hong Kong-based entity linked to such platforms.
Sources
More news
German Regulator Forces Apple to Redesign Data Consent Prompts
Apple must standardize privacy pop-ups after complaints of anticompetitive bias.
Surging Data Center Energy Demand Sparks Regulatory and Political Friction
As AI-driven power consumption threatens to overwhelm the U.S. electrical grid, industry …
Amazon Reports Record AWS Revenue Growth Amid Increased AI Capital Spending
Cloud services and massive infrastructure investments drive Amazon's latest earnings beat.
Meta, TikTok, Snap, and Google Sued Over Alleged Youth Social Media Harms
A new wrongful death lawsuit alleges major tech platforms knowingly prioritized engagemen…