OpenAI announced in its February 2025 report that it banned a cluster of ChatGPT accounts involved in a "task scam." These accounts, which appeared to originate in Cambodia, used AI to translate short comments between Urdu and English. The scam involved offering victims highly-paid jobs, then requiring them to pay their own money into a system before they could access their supposed "earnings."
Key Points
- OpenAI banned ChatGPT accounts that appeared to originate in Cambodia.
- The accounts used AI to translate messages between Urdu and English for fake-review job scams.
- Scammers posed as "recruiters" offering remote work and "mentors" guiding victims through tasks.
- The scam workflow included job postings, recruitment, reassurance, training, and demands for an "activation fee."
- Scammers occasionally used models to proofread text for spoofed luxury brand websites.
- OpenAI's policies prohibit the use of its tools for fraud or scams.
- Information about the disrupted scam networks has been shared with industry peers and authorities.
Context
According to OpenAI, the investigation into this activity began following a tip from Meta. The network primarily used OpenAI's models to translate short comments, consistent with chats between a scammer and their victims. References to messaging platforms like Telegram, WhatsApp, SMS, and various employment forums were identified, where job offers could be posted.
Why It Matters
This case highlights the use of AI in facilitating sophisticated social engineering attacks, demonstrating how language models can be leveraged to create credible-sounding communications in fraudulent schemes. For builders, it underscores the importance of integrating robust safety policies and collaboration with industry peers to counter malicious AI applications.
What To Do
- Note that OpenAI's policies strictly prohibit using its tools for fraud or scams.
- Watch for updates on OpenAI's collaborations with industry peers and authorities regarding adversarial AI behaviors.
- Review the February 2025 report from OpenAI for further details on this case study.
- Consider how similar translation capabilities could be misused in other scam types.