OpenAI announced it banned a ChatGPT account in a case study published in its February 2026 report on disrupting malicious uses of AI. The account was linked to an individual associated with Chinese law enforcement, who used the AI to plan influence activity, harassment, and online operations.
The user attempted to employ the model for covert influence operations (IO) targeting the Japanese prime minister. The model refused to assist in this planning. The user also requested the model to edit and polish status reports on what they termed “cyber special operations” (网络特战), which are covert influence operations against domestic and foreign adversaries.
Key Points
- OpenAI banned a ChatGPT account linked to an individual associated with Chinese law enforcement.
- The user attempted to plan a covert influence operation targeting the Japanese prime minister using the model.
- The ChatGPT model refused to provide assistance for the influence operation planning.
- The user asked the model to edit and polish status reports on “cyber special operations” targeting critics of the Chinese Communist Party (CCP).
- These operations involved hundreds of staff, thousands of fake accounts, and the use of locally deployed AI models like DeepSeek and Qwen.
- Evidence of the planned operation, including specific hashtags, was found on platforms like X, Pixiv, and Blogspot from late October 2025.
- The content identified across the internet did not appear to have been generated using OpenAI's models.
Context
According to OpenAI, the user's activity provided insight into a strategy of “cyber special operations” implemented by Chinese law enforcement. This strategy aims to suppress dissent and silence critics both online and offline, domestically and internationally. The effort appears to be large-scale and resource-intensive, involving hundreds of staff and thousands of fake accounts across numerous platforms. It also incorporates locally deployed AI models and a playbook of tactics ranging from abusive reporting of social media accounts to forging documents and impersonating officials.
The user's activity spanned both operational planning and tactical reporting. For instance, in mid-October, the user asked ChatGPT to help plan an operation to discredit Japanese politician Sanae Takaichi. This plan included elements such as posting negative comments, criticizing her stance on foreign immigrants using fake email accounts, and stirring up anger against U.S. tariffs. The model refused to provide advice for this plan. However, at the end of October, the user requested the model to polish a status report on the implementation of the anti-Takaichi operation, suggesting it proceeded without ChatGPT's direct assistance.
Why It Matters
This case demonstrates how AI models can be misused for malicious influence operations, even when the models themselves refuse to participate in harmful planning. It highlights the ongoing challenge of detecting and mitigating sophisticated, state-linked online campaigns that integrate AI with human operators and other digital tools.
What To Do
- Note the specific refusal behavior of the ChatGPT model when confronted with malicious planning requests.
- Observe how the operation proceeded without ChatGPT's direct planning assistance, using other resources.
- Review the described tactics of “cyber special operations” to understand the scope of such activities.
- Consider the integration of locally deployed AI models and human operators in large-scale influence campaigns.