OpenAI has banned clusters of accounts that used its models, including ChatGPT, to support malware development, debugging, phishing, and credential theft workflows. This action, detailed in a case study from OpenAI's October 2025 report, involved Korean-language operators attempting to use AI for malware and command-and-control (C2) development.
Separately, OpenAI also banned accounts involved in activity consistent with publicly reported threat groups and PRC intelligence requirements, using AI to support phishing and scripting workflows. These activities were also documented in the October 2025 report.
Key Points
- OpenAI banned Korean-language accounts using AI for malware development support, debugging, phishing, and credential-theft workflows.
- Indicators from the Korean-language activity overlapped with a Trellix report on spear phishing campaigns against South Korean diplomatic missions, XenoRAT malware deployment, and GitHub-based C2.
- The Korean-language accounts engaged with models primarily in Korean, showing structured workflows and activity in UTC+8 and UTC+9 time zones.
- OpenAI also banned accounts that supported phishing and scripting workflows, with hallmarks consistent with PRC intelligence requirements.
- These PRC-linked actors used ChatGPT to generate phishing content in multiple languages, including Chinese, English, and Japanese, and to develop tools and malware.
- The PRC-linked actors requested code snippets for tasks like testing encrypted transports, enumerating processes, and developing basic C2 prototypes.
- OpenAI found no evidence that model access enabled novel capabilities beyond what is publicly available in either case.
Context
According to OpenAI, the Korean-language operators focused on specific use cases, such as converting Chrome extensions to Safari for Apple App Store publication, configuring Windows Server VPNs, or developing macOS Finder extensions. Interactions included Windows API hooking, browser credential and cookie access workflows (DPAPI), and look-alike verification pages. Draft phishing emails in Korean, often themed around cryptocurrency, were also observed. The actors also experimented with cloud-storage services like pCloud, file.io, GDrive, and GitHub functions.
For the PRC-linked activity, OpenAI noted an overlap with threat groups tracked as UNK_DROPPITCH (Proofpoint) and UTA0388 (Volexity). The actors' playbook for creating phishing content was detailed and formulaic, often involving requests to adjust tone or add institutional references. Their development work, while technically competent, was described as unsophisticated, such as using a simple static key for AES encryption.
Why It Matters
These reports demonstrate that AI models can be misused for malicious cyber activities, prompting developers and users to understand the dual-use nature of AI capabilities. The cases highlight the ongoing need for platform providers to identify and mitigate such abuses.
What To Do
- Note that OpenAI's October 2025 report details these case studies.
- Watch for further guidance from OpenAI on identifying and mitigating dual-use activities.
- Review the types of malicious activities described, such as Windows API hooking and DPAPI credential theft, to understand potential misuse vectors.
- Observe how threat actors leverage AI for incremental efficiency in existing workflows, rather than for novel capabilities.