OpenAI has banned accounts associated with online fraud networks that leveraged AI to facilitate various scam activities. These activities included generating scam scripts, impersonating individuals, translating content, and engaging with potential victims. This case study was originally published in OpenAI's October 2025 report.
Key Points
- OpenAI banned accounts tied to online fraud networks using AI for scam scripts, impersonation, translation, and victim engagement.
- Disrupted networks likely originated in Cambodia, Myanmar, and Nigeria.
- AI was primarily used as a scaling and efficiency tool for existing scam playbooks, not to create new ones.
- Scammers used models to generate and translate correspondence, create website and social media content, and conduct basic research.
- Some scam operations used models for internal business tasks like organizing schedules and drafting announcements.
- OpenAI estimates ChatGPT is used to identify scams up to three times more often than it is used for scams.
- An OpenAI investigator used ChatGPT to identify a scam SMS message impersonating TikTok recruiters.
Context
According to OpenAI, the company is dedicated to identifying, preventing, and disrupting attempts to abuse its models for harmful ends, including scams. In the last three months, OpenAI disrupted scam networks that likely originated in Cambodia, Myanmar, and Nigeria. The abuse of models for scams ranges from lone actors to scaled operations potentially linked to organized crime groups.
OpenAI notes that scam-related activity typically follows a pattern: the "ping" (cold outreach), the "zing" (generating enthusiasm or panic), and the "sting" (extracting money or information). While scammers scatter content, AI-generated or otherwise, across messaging services and the internet, OpenAI has also observed many cases where its models have helped people identify and avoid online scams.
Why It Matters
This report highlights how AI models, while offering new efficiencies, can also be misused by malicious actors to scale fraudulent activities. It also demonstrates the potential for AI to aid in scam detection, presenting a dual challenge for builders and users in managing both the risks and protective capabilities of these technologies.
What To Do
- Note that AI is primarily used to scale existing scam playbooks, not invent new ones.
- Observe the "ping, zing, sting" pattern identified by OpenAI in scam operations.
- Consider how AI models can be used to identify scams, as demonstrated by ChatGPT's use in detecting fraudulent messages.
- Watch for further reports from OpenAI on their efforts to disrupt malicious uses of AI.