OpenAI has identified and banned accounts linked to SweetSpecter, a suspected China-based adversary, for attempting to leverage its AI models in support of offensive cyber operations. This activity, detailed in OpenAI’s October 2024 report, included researching vulnerabilities, generating code, and aiding spear-phishing campaigns. Concurrently, SweetSpecter conducted spear-phishing attacks against OpenAI employees and global government entities.
Key Points
- OpenAI banned accounts likely belonging to SweetSpecter, a suspected China-based adversary that emerged in 2023.
- SweetSpecter used OpenAI models for vulnerability research, code writing, and spear-phishing support.
- In May 2024, SweetSpecter sent spear-phishing emails to OpenAI employees, posing as ChatGPT users and including malicious attachments containing SugarGh0st RAT malware.
- OpenAI's security systems blocked these malicious emails from reaching employee inboxes.
- SweetSpecter's use of AI models did not appear to provide novel capabilities beyond publicly available resources.
- OpenAI's security teams used ChatGPT to analyze, categorize, translate, and summarize adversary account interactions.
- Another adversary, CyberAv3ngers, publicly reported as affiliated with Iran’s IRGC, also used OpenAI models for vulnerability research, code debugging, and scripting advice, as noted in the same October 2024 report.
Context
According to OpenAI, SweetSpecter's targeting previously focused on political entities in the Middle East, Africa, and Asia, with this being the first public identification of their targeting a U.S.-based AI company. The spear-phishing emails contained a malicious attachment, "some problems.zip," which, if opened, would execute SugarGh0st RAT malware designed to grant control over compromised machines. OpenAI's security team confirmed that existing security controls prevented these emails from reaching corporate inboxes. While analyzing the broader campaign infrastructure, OpenAI disrupted a cluster of ChatGPT accounts using the same infrastructure for scripting and vulnerability research tasks, mapping these interactions to LLM-themed tactics, techniques, and procedures (TTPs) proposed for the MITRE ATT&CK® Framework.
Why It Matters
This incident highlights the ongoing efforts by malicious actors to leverage AI models for cyber operations and the importance of robust security measures and threat intelligence sharing. The use of AI by adversaries like SweetSpecter and CyberAv3ngers for reconnaissance and code development underscores a persistent risk that requires continuous monitoring and defensive innovation from AI providers.
What To Do
- Note the TTPs observed from SweetSpecter, such as asking about application vulnerabilities, Log4j exploits, and internet-wide scanner creation.
- Review the examples of SweetSpecter's activities, including requests for help debugging cybersecurity tool extensions and code for programmatic text message sending.
- Observe how OpenAI leveraged its own models to analyze and summarize adversary interactions, potentially informing internal security practices.
- Compare the reported activities of SweetSpecter with those of CyberAv3ngers, noting their respective focuses on general vulnerabilities and industrial control systems.