Party Like It's 2015 (We've Seen This Movie)
Quick one before we start: we just crossed 1,500 subscribers. Didn't expect this many of you would want weekly AppSec takes in your inbox. Thanks for reading, forwarding, and hitting reply. It's the best part of doing this.
This week, instead of skimming five, we are taking a deep dive into a critical AppSec issue: The little bundles of instructions and scripts you hand an agent so it stops improvising and works from a playbook. They are genuinely useful. They are also code, plus natural language, plus whatever external link they tell your agent to go read next, all executing with your privileges after a single click of approval. Trail of Bits kicked this off by walking through five skill scanners that failed to catch serious malicious skills. Once I pulled the thread, the whole ecosystem looks like npm circa 2015, except the packages can talk your agent into things. Let's dig in.
This Week's Signals
Agent skills are now a serious AppSec risk
Trail of Bits built four deliberately malicious agent skills and ran them past every scanner meant to catch exactly this: ClawHub's detector, Cisco's skill-scanner, and all three engines behind Vercel's skills.sh (Gen, Socket, Snyk). Every scanner marked them safe. The simplest bypass used no code at all: they prepended 100,000 blank lines so ClawHub truncated the file before it ever reached the payload. Three of the four took under an hour to build, using old tricks: bytecode that diverges from its readable source, a shell script smuggled inside a .docx (which is just a ZIP), and a package-registry swap dressed up as corporate IT policy to talk the LLM judge out of flagging it.
Why this matters: A scanner is a static target, so the attacker always moves second and gets unlimited tries. This is the anchor for the whole issue: the tool everyone is bolting on as the trust decision is the weakest link in it.
Image Source: The Hacker News
Snyk audited the ecosystem and it looks like a supply chain under attack
Snyk's ToxicSkills audit is the first comprehensive read on this, scanning 3,984 skills from ClawHub and skills.sh. The findings: 13.4% (534 skills) carry at least one critical issue, 36.8% (1,467) have at least one flaw, and 76 skills shipped confirmed malicious payloads, from credential theft to backdoors. The sharpest data point is convergence: 91% of the confirmed-malicious skills combine prompt injection with traditional malware, which slips past AI safety training and static scanners at the same time. And this isn't hypothetical volume: the ClawHavoc campaign dropped 341 malicious skills into the marketplace in three days back in January.
The numbers in context: Snyk's own framing is the right one: agent skills are a software supply chain, and they need the same rigor we apply to npm, PyPI, and container registries. Right now they get almost none of it.
Image Source: Snyk
26,000 agents hijacked with one skill and one Instagram ad
This is the story that should change how you think about skill review. A research team at air.security published a clean, useful skill, got it merged into a marketplace repo with roughly 36,000 GitHub stars, then advertised it on Instagram to marketers and designers. Every trust signal cleared it: scanners (Cisco, NVIDIA, and all of skills.sh), the star count, the reputation. The trick is that the skill told the agent to read setup docs at an external URL the team controlled, so the real payload lived off to the side, invisible to any scanner that only reads the bundled files. After it spread, they swapped the doc for a malicious script and 26,000 agents ran it.
What to watch: A clean scan is a snapshot, and the ground moves. The same crew found 17,822 open-source skills (6.7M installs) exposed to this untrusted-external-instructions pattern, and a separate SkillJacking result took over 925 skills reaching 134,000 agents. If your review happens once at install, it is checking the wrong thing.
Image Source: air.security
The scanner arms race is on, and it's also crying wolf
The defensive side is not sitting still. NVIDIA's open-source SkillSpector now ships 68 detection patterns across 17 categories (prompt injection, data exfiltration, MCP tool poisoning, and more) and can run as an MCP server that gates skill installs at runtime instead of as a one-off audit. But there's a twist worth holding next to the "scanners miss everything" story: a repository-aware study of 238,180 skills found that per-marketplace scanners flag as many as 46.8% of skills as malicious, while only 0.52% stay suspicious once you check the skill against its surrounding GitHub project. So scanners miss the real attacks and drown you in false positives on the benign ones. That is the worst of both worlds, and it's why a scanner can't be the whole answer.
Why I'm flagging it: Context is the missing input. A skill that reads your SSH keys is malicious in a random marketplace download and expected in a key-rotation tool you wrote. Provenance, not pattern-matching, is what tells those two apart.
Image Source: Holzbauer et al. on arxiv
The fix is starting to look like real supply-chain hygiene
The research is converging on what actually helps, and it's not a smarter regex. OWASP now lists an Agentic Skills Top 10, with untrusted external instructions called out as a headline risk. Simon Willison's "lethal trifecta" (access to secrets, exposure to untrusted content, and a way to send data out) plus Meta's "Agents Rule of Two" (allow at most two of those three without a human in the loop) give you an actual design rule to hold a skill against. The academic taxonomy work lands in the same place: the severe problems are structural (no boundary between data and instructions, a single-approval trust model, no mandatory marketplace review) and want capability-based permission manifests, provenance, and curated catalogs, not incremental scanner tuning.
What to do with this: Treat skills like untrusted code, because that's what they are. Pull from one managed, curated source (Trail of Bits ships trailofbits/skills-curated; Anthropic supports org-managed plugins), pin versions, scope tools to least privilege, and re-check continuously instead of once.
Image Source: Li et al. on arxiv
My Take:
A Skill Is Not a File. It's Your Privileges in Someone Else's Hands.
We rebuilt the npm/PyPI supply-chain problem in about a year, and then made it harder. A skill is code and natural-language instructions and a pointer to external content, all at once, and it runs with your agent's authority the moment you approve it. That third property is the new part. Traditional packages don't rewrite their own behavior after you vet them; a skill that tells your agent to "follow the setup docs at this link" absolutely does, which is exactly how air.security turned a five-star, scanner-approved skill into 26,000 compromised agents.
Line up the evidence and the shape is clear. Trail of Bits bypassed five scanners in under an hour. Snyk found 13.4% of a 3,984-skill sample carrying critical issues and 76 shipping outright malicious payloads. A 42,000-skill study put the vulnerability rate at 26.1%. And the repository-aware work showed scanners flagging up to 46.8% of skills while only 0.52% held up under context. The detection layer is losing on both ends at once: it misses the real attacks and it cries wolf on the safe ones.
So stop trying to scan your way to trust and start managing provenance. Know where a skill came from, pin it, scope its tools to least privilege, run it against the lethal-trifecta test, and pull everything through one curated catalog you actually control instead of whatever a colleague pasted from a marketplace. This is the same design-time bet I'm making at DevArmor: decide what a component is allowed to touch based on where it came from and what it can reach, before it's inside the system, rather than hoping a scanner catches it on the way in.
A year ago "agent skills" were just a clever way to save context. Now they're a dependency tree running with your agent's keys, pulled from strangers. The one edge we have over 2015: we already know which shortcuts compound into breaches. Might as well skip them this time. See you next week.
Where I'm genuinely unsure: whether curation scales to a team pulling in a thousand skills a quarter, or whether we need a real provenance layer (signing, attestation, reputation tied to identity) that mostly doesn't exist yet. Maybe runtime gating like SkillSpector-as-MCP gets good enough to carry more of the load than I'm giving it credit for. Tell me where I've got this wrong. I read every reply.
Hacker Summer Camp: Come Watch Us Roast a CISO
We're hosting a CISO Roast at Black Hat on August 5th. A few brave security leaders on stage, taking the heat, for a good laugh and a better hallway conversation than yet another vendor pitch. If you're on the floor, stop by the DevArmor booth, #5808!
See you at Expo Hall, Booth #5808
Come heckle. Come network. Come see me argue about skills security in person. We might even rock some 2010s club bangers while we're at it.
- Amir
One click, let us know how we did.
Thanks for reading The AppSec Signal, DevArmor’s newsletter for security professionals.
Have feedback or ideas for what we should cover next?
Feel free to reach out - [email protected]
