OpenAI has banned accounts potentially associated with publicly reported Democratic People’s Republic of Korea (DPRK)-affiliated threat actors. This action follows the detection of activity involving tactics, techniques, and procedures (TTPs) consistent with groups such as VELVET CHOLLIMA (also known as Kimsuky or Emerald Sleet), and another actor linked to STARDUST CHOLLIMA (also known as APT38 or Sapphire Sleet). The information regarding these accounts was initially published in OpenAI’s February 2025 report.
Key Points
- OpenAI banned accounts potentially associated with DPRK-affiliated threat actors.
- Some banned accounts showed activity consistent with VELVET CHOLLIMA (Kimsuky, Emerald Sleet).
- Other banned accounts were potentially linked to STARDUST CHOLLIMA (APT38, Sapphire Sleet).
- The actors used OpenAI tools for coding assistance, debugging, and researching security-related open-source code.
- Activities included debugging and development assistance for Remote Desktop Protocol (RDP) brute force attacks and open-source Remote Administration Tools (RATs).
- Actors revealed staging URLs for binaries during debugging, which were shared with the security community and are now detected by vendors.
- Prompts were primarily based on existing open-source information, and model generations did not offer novel capabilities or resulted in refusals.
Context
According to OpenAI, the banned accounts primarily sought information related to cyber intrusion tools or operations. They also demonstrated interest in cryptocurrency-related topics, which aligns with the typical blend of financial and cyber activity seen in DPRK-associated threat groups. The actors utilized OpenAI models for tasks such as debugging auto-start extensibility point (ASEP) locations and techniques for MacOS.
OpenAI received a tip from a trusted industry partner that led to the detection of these accounts. The company submitted the staging URLs for binaries discovered during the actors' activities to an online scanning service, facilitating sharing with the security community. This action resulted in the binaries being reliably detected by multiple security vendors, providing protection for potential victims. OpenAI also shared the actors' payloads with the security community to further disrupt their operations.
Why It Matters
This case highlights the ongoing challenge of preventing the misuse of AI tools by malicious actors, even when the models do not provide novel capabilities. It underscores the importance of proactive monitoring, industry collaboration, and rapid response to mitigate potential threats. For builders, it illustrates the need to consider potential misuse vectors and integrate security measures into AI development and deployment.
What To Do
- Note that AI models may be used for researching existing open-source tools and techniques, even without generating novel exploits.
- Observe how prompt-based interactions, even those resulting in refusals, can still reveal malicious intent or infrastructure.
- Consider the role of industry partnerships in identifying and disrupting malicious AI-assisted activities.
- Watch for updates from OpenAI regarding their ongoing efforts to counter malicious uses of AI.