AI Security Gets Real: From Supply Chain Worms to Model Theft

Page content

AI Security Gets Real: From Supply Chain Worms to Model Theft

The AI security conversation just shifted from theoretical to painfully practical. While we’ve been debating governance frameworks and ethical guidelines, attackers have been busy figuring out how to weaponize AI systems, steal model capabilities, and turn our shiny new AI assistants against us.

This week brought a perfect storm of AI-related security incidents that should make every CISO sit up and pay attention. We’re not just talking about prompt injection anymore – we’re dealing with sophisticated supply chain attacks that specifically target AI systems and nation-state actors stealing AI model capabilities at scale.

The NPM Worm That Poisons AI Assistants

Let’s start with the most creative attack I’ve seen in months. Security researchers discovered a new supply chain attack called ‘Sandworm_Mode’ hitting NPM packages. This isn’t your typical malware – it’s designed specifically for our AI-integrated development environments.

Here’s what makes this particularly nasty: the malicious code doesn’t just exfiltrate secrets and propagate like a traditional worm. It actively poisons AI coding assistants. Think about that for a second. Your developers are using Copilot or similar tools to write code, and now the AI is suggesting malicious patterns because it’s been fed poisoned data.

The attack also includes a “destructive dead switch,” which means the attackers can flip a switch and cause widespread damage across infected systems. We’ve seen supply chain attacks before, but this level of AI-specific targeting represents a new evolution in threat actor tactics.

Chinese Firms Caught Red-Handed Stealing Claude

Meanwhile, Anthropic publicly accused three Chinese AI companies – DeepSeek, Moonshot, and MiniMax – of using distillation attacks to steal capabilities from their Claude AI model. This isn’t just corporate espionage; it’s a systematic effort to extract the intellectual property baked into trained models.

Distillation attacks work by repeatedly querying a target AI model and using those responses to train a competing model. It’s like reverse engineering, but for neural networks. The accused companies were essentially using Claude as an unpaid teacher to train their own models, stealing years of research and millions of dollars in training costs.

What’s particularly concerning is that this represents state-level AI espionage. These aren’t random hackers – they’re well-funded organizations with the resources to conduct large-scale model theft operations.

Microsoft Finally Takes Copilot Data Seriously

On a more positive note, Microsoft is expanding data loss prevention controls to prevent Copilot from processing confidential documents across all storage locations. This should have been day one functionality, but better late than never.

The new controls let administrators block Copilot from accessing sensitive Word, Excel, and PowerPoint files regardless of where they’re stored. This addresses one of the biggest concerns we’ve had about AI assistants – the risk of confidential data being processed and potentially leaked through AI interactions.

For organizations already using Microsoft 365 Copilot, this is a must-implement control. The last thing you want is your AI assistant accidentally exposing trade secrets or regulated data in its responses.

The Identity Risk Calculation Problem

Here’s something that’s been bugging me for months, and a recent article on identity prioritization nails exactly why. Most organizations are still treating identity risk like IT tickets – prioritizing by volume or whoever screams loudest.

But in environments where AI systems have identities, service accounts proliferate, and non-human entities outnumber humans, that approach falls apart completely. We need to think about identity risk as a compound calculation involving control posture, hygiene, business context, and intent.

This is especially critical when AI systems have privileged access to data and systems. A compromised AI service account isn’t just another identity incident – it’s potentially a data exfiltration goldmine.

Making AI Decisions Auditable

Finally, there’s growing recognition that AI decisions need to be provable, not just visible through dashboards. When an AI system makes a security decision – blocking a transaction, flagging a user, or granting access – we need a complete audit trail of how that decision was made.

This isn’t just about compliance (though that’s important). It’s about being able to debug AI systems when they go wrong and proving to regulators, auditors, and incident response teams that our AI-driven security controls are working as intended.

The Bottom Line

We’re past the point where AI security is a nice-to-have or a future concern. Attackers are already weaponizing AI systems, nation-states are stealing model capabilities, and our AI assistants are becoming prime targets for sophisticated attacks.

The good news is that we’re also seeing vendors step up with better controls and the security community starting to take AI-specific risks seriously. But we need to move faster. Every day we delay implementing proper AI security controls is another day we’re giving attackers a head start.

Sources