OpenAI has taken action against a cluster of activity that leveraged its API to produce comments critical of the Anti-Corruption Foundation, established by the late Alexei Navalny. This operation, dubbed “Corrupt Comment” by OpenAI, involved generating English-language criticisms that were subsequently posted on X by fake accounts.
This case study was initially published in OpenAI’s October 2024 report.
Key Points
- OpenAI banned a small cluster of activity using its API to generate comments posted on X.
- The comments typically criticized members of the Anti-Corruption Foundation, founded by Alexei Navalny.
- The operation, named “Corrupt Comment,” used OpenAI models to generate English-language comments.
- These comments were posted by fake accounts on X, often as replies to posts in Russian.
- The activity focused on senior figures at the Anti-Corruption Foundation (FBK) and Navalny’s associates.
- The X accounts posting these comments were mostly created in December 2023, had zero followers, and received no replies.
- Some accounts used profile pictures of scenery or fake profile pictures with indicators of AI-generated adversarial networks (GAN) from as far back as 2019.
Context
According to OpenAI, the operator or operators used its API to generate comments questioning the FBK's transparency. This approach is described as “theme and variations,” where numerous posts conveyed the same core message. OpenAI identified this activity on X, noting that most accounts had no followers or replies.
Why It Matters
This action demonstrates OpenAI's ongoing efforts to detect and mitigate malicious uses of its AI models. It highlights the challenges of preventing the misuse of generative AI for coordinated influence operations, even when the immediate impact on conversation volume is limited.
What To Do
- Note OpenAI's assessment of the activity's impact using the Breakout Scale, rating it as 1 (lowest).
- Observe that the operation did not appear to use models to analyze social media content or generate relevant responses.
- Review the characteristics of the fake accounts, such as creation date, follower count, and engagement levels.
Keep Exploring
/atlas/claude-family /atlas/gpt-family /techniques/system-user-separation
