OpenAI has published a case study detailing its actions against a cluster of ChatGPT accounts that were used to support cyber operations. The activity, which overlapped with publicly reported threat groups, showed characteristics consistent with the intelligence requirements of the People's Republic of China (PRC). This case study was originally included in OpenAI's October 2025 report.
The threat actors primarily used the models to gain incremental efficiency in existing workflows, focusing on crafting phishing content and debugging or modifying their tooling. OpenAI stated that its models did not introduce novel offensive capabilities.
Key Points
- OpenAI identified and banned a cluster of ChatGPT accounts linked to threat groups UNK_DROPPITCH (Proofpoint) and UTA0388 (Volexity).
- The activity displayed hallmarks consistent with PRC intelligence requirements, including Chinese language use and targeting Taiwan's semiconductor sector, U.S. academia, think tanks, and groups critical of the CCP.
- Actors used ChatGPT to generate phishing content in multiple languages (Chinese, English, Japanese) and to develop tools and malware.
- The model assistance focused on linguistic fluency, localization, and persistence, rather than enabling new offensive capabilities.
- The actors' development work, while technically competent, was described as unsophisticated, such as discussing AES for C2 traffic but using a simple static key.
- Activities mapped to LLM ATT&CK categories including LLM-Optimized Payload Crafting, LLM-Enhanced Anomaly-Detection Evasion, and LLM-Assisted Social Engineering.
- OpenAI disabled all associated accounts and shared relevant indicators with industry partners.
Context
According to OpenAI, the threat actors used ChatGPT for two main tasks: generating content for phishing campaigns and assisting with tool and malware development. Their phishing content generation involved creating concise, formally polite emails from academic, industry, or conference personas, often requesting tone adjustments, regional term swaps, or specific institutional references. For tool development, they requested code snippets and checklists for routine tasks, such as testing encrypted transports, enumerating processes, or gathering environment details. They also sought suggestions for basic obfuscation and operational security (OPSEC) tweaks, like renaming functions or hiding strings, and pushed towards low-to-mid maturity command-and-control prototypes.
Some of the actors' Go-based development activities showed overlap with malware tracked as GOVERSHELL (Volexity) or HealthKick (Proofpoint). The operators also explored further automation using DeepSeek, resembling exploratory work for mass phishing automation, though OpenAI could not independently confirm if such automation proceeded or which model was ultimately used.
Why It Matters
This report illustrates how threat actors are leveraging large language models to enhance the efficiency and sophistication of existing cyber operations, particularly in areas like social engineering and malware development. For builders and security researchers, it highlights the importance of understanding how LLMs can be integrated into malicious workflows, even without introducing novel offensive capabilities, and the need for robust detection and disruption strategies.
What To Do
- Note the specific threat group names (UNK_DROPPITCH, UTA0388, GOVERSHELL, HealthKick) and their associated activities for threat intelligence correlation.
- Review the described LLM ATT&CK categories to understand how models are being used in various stages of cyber operations.
- Observe the examples of phishing content generation and tool development to identify patterns that could indicate malicious use of LLMs.
- Consider the reported focus on incremental efficiency, linguistic fluency, and localization as key motivations for threat actors using LLMs.