OpenAI rolls out CSAM detector and ChatGPT warnings for child abuse content
OpenAI launched a new image classifier that detects child sexual abuse material, including AI-generated images, and is handing select companies free API access. ChatGPT now warns users who try to sexualize children and suspends their accounts for 24 hours. It follows lawsuits and safety criticism aimed at the company's tech.
- 2 sources report it: techcrunch.com, cryptobriefing.com.
- The source published it on Nov 16, 2026 at 5:00 PM.
Nothing open. Every check passed.
OpenAI rolled out a new set of tools on Tuesday aimed at stopping child sexual abuse material (CSAM), including an image classifier that sniffs out abusive images — even ones made by AI. Select companies get free access to the classifier through OpenAI's API, and it's built to spot visual signs of abuse in images it has never seen before, scoring them to steer human reviewers.
The classifier was developed with the National Center for Missing and Exploited Children (NCMEC), and OpenAI is teaming up with Tech Coalition and ROOST to widen access. Early partner testing found it flags 48% fewer images while surfacing the same number of abusive ones, cutting reviewer workload, and images it flags are twice as likely to actually be abusive, according to TechCrunch.
Inside ChatGPT, users who try to generate or access content that sexualizes children will now hit a warning screen. The app points them toward specialized support, and the account gets suspended for 24 hours as a warning. CSAM content and related requests were already blocked; OpenAI says repeated offences can mean permanent account removal.
OpenAI says it shaped those deterrent messages with Protect Children, the Lucy Faithfull Foundation, and Joel Scanlan at the University of Tasmania, who independently evaluated a trial of the warnings. The company is also publishing developer guidelines for preventing, detecting and reporting CSAM, and says early testing pushed several developers to adopt child safety tooling and register to report CSAM to NCMEC. Law enforcement gets its own package: dedicated plugins, investigative tools and complex data analysis, with access managed by program administrators, plus investigator training and a field guide.
The push lands against a rough backdrop. OpenAI faces multiple lawsuits blaming its technology for harming users, particularly children, and nonprofit Common Sense Media has labeled its tech an 'unacceptable risk' for teens. Rival chatbots — Grok in particular — have been used to create nonconsensual sexualized imagery, including of children.
It's the latest in a run of safety moves from OpenAI over the past 12 months: a safety blueprint, updated content guidelines barring inappropriate material, model retuning to block harmful generations, and a special version of ChatGPT built for teens.
TopicsOpenAI · ChatGPT · National Center for Missing and Exploited Children · Tech Coalition · ROOST · Common Sense Media · Grok · Lucy Faithfull Foundation
Written by AI from verified reporting.

