OpenAI has banned accounts linked to Chinese government entities for using ChatGPT to support surveillance-related planning, targeted profiling, and research on critics and other individuals. This activity was detailed in OpenAI's October 2025 report on disrupting malicious uses of AI. The company also reported on similar disruptions of influence operations linked to Israel and Russia in May 2024 and February 2026, respectively.
Key Points
- OpenAI banned PRC-linked accounts that used ChatGPT for surveillance-related planning, targeted profiling, and research on critics.
- This activity was consistent with individual users, not large-scale institutional adoption of OpenAI's models.
- Some banned accounts attempted to use ChatGPT to develop tools for large-scale monitoring, such as social media listening tools.
- One user asked ChatGPT to design promotional materials for a social media 'probe' tool capable of scanning platforms like Twitter/X, Facebook, Instagram, Reddit, TikTok, and YouTube.
- Another user sought assistance from ChatGPT to write a proposal for a 'High-Risk Uyghur-Related Inflow Warning Model'.
- Other PRC-linked accounts used ChatGPT for targeted profiling and online research, such as identifying funding sources for critics or organizers of petitions.
- OpenAI also identified and banned accounts linked to an Israel-origin operation dubbed "Zero Zeno" in May 2024, which used AI to generate political content.
- A Russia-linked operation called "Trolling Stone" was disrupted in February 2026 for using AI to generate social media comments about a Russian cult leader's arrest.
Context
According to OpenAI, these disruptions align with its commitment to preventing AI tools from being used by authoritarian regimes to amass power and control citizens. The company states that its investments in detecting and disrupting such abuses provide insights into how future AI capabilities might be misused. The PRC-linked activity appeared aimed at supporting large-scale monitoring of online or offline traffic and more in-depth profiling, according to OpenAI.
Why It Matters
This information highlights the ongoing challenges in preventing the misuse of AI technologies for surveillance and influence operations. It underscores the need for developers and deployers to implement robust safeguards and monitoring mechanisms to detect and mitigate such activities, protecting individuals from potential harms.
What To Do
- Note the specific examples of misuse, such as designing surveillance tools or targeted profiling, to understand potential abuse vectors.
- Review OpenAI's October 2025 report for further details on the PRC-linked activity.
- Examine the May 2024 and February 2026 reports from OpenAI to compare patterns of influence operations across different actors.
- Consider how current and future AI capabilities could be adapted for similar malicious purposes.
