OpenAI has identified and banned accounts linked to an Israel-origin influence operation it named "Zero Zeno." This operation utilized AI models to create and disseminate content across various platforms, including X, Facebook, Instagram, and dedicated websites. The activity was first detailed in OpenAI's May 2024 report on disrupting malicious uses of AI.
The "Zero Zeno" operation generated content that was anti-Hamas, anti-Qatar, pro-Israel, anti-BJP, and pro-Histadrut. It targeted audiences in Canada, the United States, and Israel with English and Hebrew content, and began targeting India with English-language content in early May. The operation also appeared to be preparing a campaign for audiences in Ghana.
Key Points
- OpenAI banned accounts linked to the "Zero Zeno" operation, which originated in Israel.
- The operation used AI models to generate web articles and social media comments posted across X, Facebook, Instagram, and websites.
- Content themes included anti-Hamas, anti-Qatar, pro-Israel, anti-BJP, and pro-Histadrut messaging.
- Targeted audiences were in Canada, the United States, Israel, and, from early May, India.
- The operation also used AI to create fictional social media personas and conduct research into individuals commenting on the Histadrut trade union.
- Some social media accounts used profile pictures that appeared to be generated by earlier AI, specifically generative adversarial networks (GAN).
- OpenAI assessed "Zero Zeno" as a Category 2 operation on the Breakout Scale for influence operations, indicating low engagement.
Context
According to OpenAI, the "Zero Zeno" network was operated by STOIC, a political campaign management firm in Israel. The operation used OpenAI's models to generate content and create fictional personas for social media. The content was distributed across multiple social media platforms and four websites that posed as activist groups. OpenAI noted that the operation sometimes used its own fake accounts to comment on its posts, creating an artificial impression of engagement. The Atlantic Council’s Digital Forensic Research Lab (DFRLab) identified some of this content on Facebook, Instagram, and X.
Why It Matters
This case illustrates how AI models can be misused for influence operations, prompting builders and deployers to consider the ethical implications and potential for abuse of their technologies. It highlights the ongoing need for platforms to detect and disrupt coordinated inauthentic behavior, even when engagement levels are low.
What To Do
- Review OpenAI's May 2024 report on disrupting malicious uses of AI to understand the full scope of the "Zero Zeno" operation.
- Note the types of content generation and persona creation capabilities that were exploited in this influence campaign.
- Examine the methods used by OpenAI to identify and ban accounts associated with such operations.
Keep Exploring
/atlas/claude-family /atlas/gpt-family /techniques/role-objective /techniques/constraints /techniques/output-schema /techniques/system-user-separation /studio?pack=foundation