Meta has faced a serious ethical crisis after failing to remove ads containing AI-generated child sexual abuse material (CSAM) on Facebook and Instagram. The Tech Transparency Project (TTP) revealed that out of 332 CSAM ads detected this year, the vast majority promoted apps made in China, with many featuring so-called 'nudify' apps that can easily alter children's images. One particularly disturbing ad animated a young royal’s image into performing a graphic sex act, while another turned a 14-year-old Instagram influencer’s photo into a video of oral sex. The ads, which often enticed viewers with promises of 'dark, seductive' videos, also depicted preteen girls predominantly. Such apps are not only illegal but also deeply troubling, highlighting the urgent need for stronger regulation.

These ads are a stark reminder of how AI can be weaponised to harm the most vulnerable, with real kids' images being manipulated and exploited. The Justice Department has already clarified that AI CSAM is just as harmful as traditional CSAM, making the case for stringent measures to prevent such exploitation more pressing than ever. This raises serious questions about the ethical use of AI technology and the responsibility of tech companies to protect their users.

While the incident has sparked outrage, it also underscores the complex challenges faced by social media platforms in policing content, especially when dealing with evolving and sophisticated technologies. The TTP’s findings suggest that Meta’s systems are struggling to keep up with the sheer volume and complexity of ads, let alone those that are designed to evade detection. This crisis is not just about individual apps but about the broader implications of AI in our society.

The incident has prompted calls for greater transparency and accountability from tech companies, but also from governments and international bodies. There is an urgent need for a coordinated global approach to address the growing threat of AI-generated CSAM. The future of AI technology will depend on how effectively we can balance innovation with ethical considerations, and protect the privacy and safety of individuals, especially children, online.