OpenAI has identified and banned a network of accounts linked to a Russia-origin operation named "Bad Grammar." This network utilized OpenAI's models to generate comments for Telegram, focusing on political discussions in English and Russian.
This activity was first detailed in OpenAI's May 2024 report. The operation targeted audiences in Russia, Ukraine, the United States, Moldova, and the Baltic States.
Key Points
- OpenAI banned accounts linked to the "Bad Grammar" operation, which originated in Russia.
- The operation used OpenAI's models to generate English- and Russian-language comments for Telegram.
- Content focused on Ukraine, Moldova, Baltic states, and US politics.
- Operators used OpenAI models to debug code for automating Telegram posting and to generate comments in response to specific posts.
- The network employed at least a dozen Telegram accounts to post these comments.
- English comments adopted various fake personas, while Russian comments were more thematic.
- The campaign primarily commented on posts from channels such as @Slavyangrad, @police_frequency, and @SGTNewsNetwork.
- The operation's impact was assessed as Category 1 on the Breakout Scale, indicating limited amplification and engagement.
Context
According to OpenAI, the "Bad Grammar" network established a comment-spamming pipeline. Operators first used OpenAI models to debug code intended for automated Telegram posting. They then generated comments in Russian and English, which were subsequently posted using multiple Telegram accounts. The network's name reflects its repeated posting of ungrammatical English and its struggle to build an audience.
Why It Matters
This case demonstrates how AI models can be misused for influence operations, even if the operations themselves achieve limited impact. It highlights the ongoing challenge for AI developers to detect and mitigate such activities, requiring continuous monitoring and policy enforcement.
What To Do
- Note that OpenAI's May 2024 report provides further details on this and similar incidents.
- Watch for updates from AI providers regarding their efforts to counter malicious uses of AI.
- Review the Breakout Scale for assessing the impact of influence operations.
Keep Exploring
/atlas/claude-family /atlas/gpt-family
