You Can't Tell What an Agent Skill Will Do by Reading It with Roey Ben Chaim, Zenity

Agent skills cannot be trusted or verified through static analysis alone because malicious payloads are often hidden behind remote fetches, obfuscation, or s

You Can't Tell What an Agent Skill Will Do by Reading It

Abstract

Agent skills - modular, capability-on-demand bundles of instructions, resources, scripts, and executable code - have become a primary extension mechanism for autonomous agents. Because skills are authored by third parties, they constitute an untrusted input to a privileged execution context. This synthesis examines why static analysis of skill manifests fails to establish trustworthiness and documents an alternative methodology: a detonation chamber that executes skills under instrumentation, adapted from malware research practice. The architecture combines automated environment synthesis, trajectory generation, adversarial prompting to defeat refusal behavior, and multi-layer telemetry spanning agent traces, process creation, file system mutation, and network capture. Applied at scale, the approach surfaced 100 malicious skills among thousands tested and characterized a coordinated supply chain campaign whose command-and-control infrastructure was associated with 1.7 million downloads. The findings indicate that skills must be governed as package dependencies rather than as configuration.

1. Introduction

A skill supplies an agent with capability on demand: natural-language instructions, supporting resources, helper scripts, and code that the agent may load and execute when a task appears to require it. The design is deliberately open - skills are written by third parties - and that openness is the root of the security problem addressed here.

The central difficulty is not detection of unusual behavior but attribution of intent. An official Azure diagnostic skill was observed traversing cloud configuration, Kubernetes configuration, the identity service, and the instance metadata endpoint. The observable action sequence is indistinguishable from that of an information stealer; only context - the skill's stated purpose, provenance, and the task that invoked it - separates legitimate diagnostics from credential theft.

"It was an official Azure diagnostic skill. And this is exactly the point where it's hard for us to identify whether a skill is malicious or benign."

The thesis advanced is that skills cannot be verified through static inspection alone, because malicious payloads are routinely displaced behind remote fetches, obfuscation, or social engineering of the analyzer itself. Reliable detection requires executing the skill and observing what actually occurs. Section 2 establishes analytic context; Section 3 presents the failure modes of static scanning, the detonation chamber architecture, and empirical case studies; Section 4 distills implementation findings; Sections 5 and 6 discuss implications for agent supply chain security.

2. Background and Related Work

Three strands of prior work inform the approach. First, malware detonation - executing a suspect artifact in an instrumented sandbox to recover behavior that resists disassembly - is long-established practice in malware research and is the technique borrowed and adapted here for agents and skills.

Second, research by Trail of Bits demonstrated that large language model (LLM) based analyzers are themselves attackable. A skill that manipulated a package registry carried a brief source comment asserting that the "OpSec team approved this change." The assertion was sufficient to defeat the analyzer's judgment, illustrating that an LLM reviewing adversarial text operates inside the attack surface rather than outside it.

Third, a proof-of-concept by Zach Hoffman established the multi-stage pattern: a skill presented as a "security review tool" installed a second skill, which retrieved hidden instructions from a skill.md hosted on GitHub and executed a payload. No stage of that chain is visible to a scanner that reads only the first manifest. Together these results motivate an agent supply chain security model in which skills are treated as dependencies with transitive closure, not as inert prompt fragments.

3. Core Analysis

3.1 Why Static Scanning Fails

Static scanners read skill.md and adjacent files, but the executable payload frequently lives elsewhere. A manifest may act merely as an entry point that invokes a remote endpoint, an installable package, or an install script, each of which can fetch further payloads at runtime. A scanner limited to the entry file therefore misses the full payload chain unless it has a mechanism to track every downstream fetch and installation it triggers.

This limitation compounds with the discovery that LLM-based analyzers can be socially engineered using the same techniques effective against human reviewers - in the Trail of Bits case, a single comment claiming prior approval. Because neither traditional pattern-matching scanners nor LLM-based reviewers proved bulletproof, the research team turned to dynamic execution, or detonation, as the primary detection mechanism rather than a supplementary one.

3.2 Building the Detonation Chamber

The chamber evolved through several iterations to address practical obstacles. The initial version (v1) ran a task prompt, an agent, and the target skill together inside a sandbox over a span of minutes to hours, observing only the agent's own traces.

Three deficiencies emerged and were resolved iteratively:

• Missing environment. Many skills depend on external services or CLIs (a Notion skill, for example, requires a Notion-like service to be present). This was solved with a world builder component that constructs an environment appropriate to the skill under test.
• Generic task prompts. A single task prompt cannot exercise all skill types - presentation, deployment, and brainstorming skills each require distinct prompts and route coverage. An operator component was added to analyze each skill and determine the set of trajectories needed to invoke its full behavior.
• Safety refusals. Built-in agent guardrails caused refusals to decode or execute even benign-looking payloads, suppressing the very behavior the chamber needed to observe. This was addressed with an adversarial operator that persuades the agent to proceed and restarts failed runs.

"We had to give him an offer it can't refuse and essentially create an adversarial operator that convinces the agents to run."

The final capturing mechanism records process creation events, file system events, and network packet captures in addition to agent traces, because side effects such as npm post-install scripts spawning child processes are invisible from the agent's own trace log alone.

3.3 Case Studies from the Detonation Chamber

The Zach Hoffman proof-of-concept skill, posing as a "security review tool," installed a second skill that fetched hidden instructions from a GitHub-hosted skill.md and executed a payload. The chain was undetectable by static scanners and only recoverable by executing it end to end.

Running the chamber at scale on the DotA stage framework, the research team uncovered 100 malicious skills out of thousands tested. One example involving Polymarket contained an install script referencing a non-official package ("Polymarket Club Client V2") pointing to an unofficial npm registry; the package was in fact an information stealer.

3.4 The Paperclip Supply Chain Attack Campaign

A coordinated campaign, attributed to an actor identified as "Carly"/"Carly Paperclip," created a fake GitHub organization mimicking a legitimate open-source orchestration project, registered a lookalike domain, and cloned the project. A malicious "log action" call was inserted across all project roles, fetching a payload from an attacker-controlled endpoint, decoding it, saving it to a temporary file, executing it via Node, and self-destructing after 60 seconds. The payload was a credential harvester scanning SSH, AWS, and Azure credential paths and posting exfiltrated data to an attacker feedback endpoint.

The same attack pattern recurred in a Browserify-based Python package and in a "PaperClip AI" Python variant; both were removed from PyPI within roughly one to two hours of reporting. Despite these takedowns, variants sharing the same command-and-control infrastructure were trending and had reached 1.7 million total downloads. The incident was reported to Vercel and Microsoft, and the associated infrastructure was taken down in under 12 hours.

4. Technical Insights

The detonation chamber architecture illustrates several transferable implementation principles. First, environment fidelity matters: a world builder that approximates the services a skill expects to interact with is necessary to elicit genuine behavior rather than error-path behavior. Second, task coverage must be tailored per skill category, since a generic prompt under-exercises skill types with divergent operational purposes. Third, safety guardrails inside the agent under test are a confound for security researchers as much as a protection for end users, requiring an adversarial operator to obtain representative execution traces - a tension worth flagging for teams building similar harnesses. Fourth, telemetry must span multiple layers (process, file system, network, agent trace) because no single layer captures all side effects; npm post-install scripts, for instance, spawn processes invisible to agent-trace-only monitoring.

Attacker techniques observed through the chamber generalize into reusable detection heuristics: persistence via writes to settings files (solo.js/solo.md/agent.md) that re-trigger reinstallation; silent displacement of trusted components (a fake "skill creator" uninstalling the official one before installing itself); remote prompt execution where the initial prompt omits content and fetches it from a remote location subject to later takeover; and hook abuse, where a skill registers against triggers such as "user prompt submit" to fire on every subsequent user message.

5. Discussion

These findings situate skills within a broader agent supply chain rather than as an isolated risk category. Any content entering agent context - websites, database rows, third-party packages, even incidental text such as video comments - constitutes a comparable risk vector, since the underlying failure mode (untrusted content influencing privileged execution) is identical regardless of the delivery channel.

A notable open question is scalability: dynamic detonation is resource-intensive relative to static scanning, and the chamber's reliance on an adversarial operator to defeat safety refusals raises a dual-use concern, since the same bypass techniques could inform jailbreak research outside the security-testing context. The speed of the reported takedowns (one to two hours for PyPI packages, under 12 hours for hosting infrastructure) suggests that platform-level response capacity is currently adequate for isolated incidents but has not been tested against campaigns designed for persistence across many independently registered artifacts, as the 1.7 million download figure implies.

6. Conclusion

This analysis demonstrates that static review of agent skills - whether performed by pattern-matching scanners or LLM-based analyzers - cannot reliably establish trustworthiness, because payload chains are routinely displaced behind remote fetches and because analyzers themselves are susceptible to social engineering embedded in the content under review. A detonation chamber combining environment synthesis, adversarial trajectory generation, and multi-layer telemetry recovered malicious behavior invisible to static methods, surfacing 100 malicious skills in one deployment and tracing a supply chain campaign with 1.7 million affected downloads.

The practical takeaway for engineering teams adopting agent skills is to treat them as package dependencies: inspect the full transitive payload chain, apply least-privilege execution, and favor runtime observation over manifest review when provenance cannot be fully verified.

Sources

• You Can't Tell What an Agent Skill Will Do by Reading It - Roey Ben Chaim, Zenity - Original Creator (YouTube)
• Analysis and summary by Sean Weldon using AI-assisted research tools

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub