Last reviewed: 2026-08-29

Direct answer

For coding agent skills security, treat every reusable Agent Skill as an executable supply-chain input, even when its main file is Markdown. Admit it only after five gates: package shape, source and integrity, static safety, isolated behavior, and task usefulness. A green text scan is evidence for one gate, not a trust decision.

The reason is architectural. Google’s explanation of Agent Plugins 1.0 describes a deliberately small, vendor-neutral package format: a manifest, fixed skill directories, and optional MCP configuration. It also explicitly leaves installation, permission models, sandboxing, trust, provenance verification, and user experience to clients and operators. GitHub’s Agent Plugins 1.0 announcement shows why this boundary matters: one package can travel across compatible clients and marketplaces, while managed settings can enable or block plugins and restrict marketplaces. Portability broadens the places a package can be loaded; it does not supply the missing controls.

Build the gate as a fail-closed admission workflow:

  1. Describe the artifact. Record the publisher, source location, reviewed revision, digest, package layout, included scripts, MCP entries, requested file paths, network destinations, and expected task boundary. A missing declaration is an unknown capability, not an implied permission.
  2. Inspect without executing. Parse manifests and front matter, enumerate ordinary and hidden files, lint scripts, inspect dependency references, and scan for prompt injection, data-exfiltration behavior, secrets, PII, licensing problems, and suspicious encoding. Keep findings attached to file paths and line numbers.
  3. Observe in isolation. Run the skill in a disposable sandbox with no production files, no ambient sensitive data, and network denied by default. Capture reads, writes, process launches, attempted destinations, and tool calls. Allow an exception only when the declared task requires it and the destination is explicitly approved.
  4. Compare usefulness. Run the same happy, negative, and irrelevant cases with and without the skill, using the same prompt, model, inputs, and grading rules. Require a bounded improvement on the intended task without a safety regression or surprising side effect.
  5. Approve and pin. A curator or security owner records the decision, pins the reviewed revision and digest, applies client policy, and defines a revocation path. Re-run the gate whenever the artifact, a bundled server, a dependency, or the client policy changes.

This combines static and behavioral evidence without pretending that either is perfect. It also gives operators a useful answer to why a skill was installed that can be reconstructed later.

Who this is for

This is for platform engineers, security engineers, developer-experience teams, and repository owners who let coding agents load skills from an internal catalog or a public marketplace. It is especially useful when one skill is bundled with scripts or an MCP server, when several clients consume the same package, or when an agent can read and modify valuable workspaces.

Individual developers can use a smaller version: stage the package in a disposable directory, inspect every file, run a few synthetic tasks with network disabled, and ask a second person to review the capability map. Teams should centralize the evidence and policy so that a hurried install does not bypass the same checks applied to a larger plugin.

Key takeaways

  • A portable plugin format is a packaging contract, not a permission or trust system.
  • Require a source reference and immutable digest before a package reaches a client.
  • Static checks should cover structure, scripts, prompt-injection and exfiltration patterns, secrets and PII, licenses, and hidden or generated files.
  • Behavioral tests belong in an isolated sandbox. Observe data flow and operating-system effects, not only the words in SKILL.md.
  • Compare with-skill and without-skill runs on the same cases. Measure correctness and task completion alongside side effects, token use, and time.
  • Keep approval, client allowlists, and revocation separate from the package author’s claims.
  • Store sanitized evidence that lets an operator explain an approval without retaining user prompts or file contents.

Sources checked

The following five public sources informed this guide and support the factual claims above and below.

Contract details to verify

Start with a contract that a machine can validate and a human can review. For an Agent Plugin, verify the fixed locations described by Google and GitHub: plugin.json, one directory per skill under skills/, and mcp.json for servers, plus any client-specific namespace. Check that each MCP entry has an explicit type and that a failed optional server cannot silently change the intended skill behavior. If you are shipping one skill with no companion component, the Google source notes that a plugin may be unnecessary; use the simpler package when it is sufficient.

Keep the contract separate from the scanner output. The following is a sanitized record shape, not an authorization grant:

package_name: example-skill
source_ref: approved-ref
artifact_digest: recorded-out-of-band
components:
  - skills/example/SKILL.md
entrypoints:
  - skills/example/SKILL.md
requested_paths:
  - workspace/read-only
requested_network: none
declared_commands:
  - lint
test_cases:
  - happy-path
  - negative-path
  - irrelevant-task
expected_effect: bounded task improvement
decision: pending-review

The operator workflow should make both outcomes concrete.

Happy path. Fetch the package into a staging area that cannot see developer home directories or build credentials. Resolve the reviewed source revision and calculate the digest before opening a client. Parse the manifest, enumerate all files, and match every entrypoint to the declared capability map. Static checks return no blocking finding; the script and license checks pass; and the requested file and network scope is narrow enough to approve. Run a small task matrix in the sandbox. On the intended task, the with-skill run produces the expected artifact, uses only the permitted workspace path, makes no network request, and improves the chosen quality measure without a safety regression. The reviewer records approval, pins the digest, publishes the package to the approved catalog, and enables it only for the clients and teams that need it.

Error path. If JSON is malformed, a skill directory is missing, a script appears only in a hidden directory, or an MCP entry has an undeclared transport, stop before execution and quarantine the package. If static analysis finds an injection or exfiltration pattern, a license conflict, PII, or an unexplained download, reject or send it to manual review. If the sandbox observes a write outside the declared workspace, an attempted network destination, a process launch not in the contract, or sensitive data flowing to an output, fail the gate even if the final answer looks correct. If the with-skill run does not improve the intended case, harms an irrelevant case, or behaves differently from the reviewed revision, do not install it; preserve the evidence and ask the publisher for a new revision.

Sanitized logs make this workflow auditable without becoming another data leak. Keep fields such as event_id, package_name, source_host, source_ref, artifact_digest, client, sandbox_id, case_id, phase, declared_capabilities, observed_capabilities, network_decision, file_access_summary, finding_codes, outcome, duration_ms, runner_version, and policy_version. Store counts, destinations, classifications, and hashes or lengths instead of file contents, prompts, environment values, or full command output. Redact at collection time, not during a later export. Give each run a retention period and restrict who can read the evidence.

NVIDIA’s SkillEvaluator methodology is a useful model for separating questions. Its first tier checks schema, quality, security, secrets and PII, licenses, and scripts; its second looks for duplicated or overlapping guidance; its third runs controlled live tasks in isolated environments with and without the skill. Adopt the separation even if you implement the checks yourself. The point is to know whether a failure is structural, security-related, redundant, or behavioral.

For governance, connect the decision to a registry record rather than to a marketplace description alone. AWS describes ARD and an Agent Registry workflow in which resources are cataloged and approval or authorization settings can be applied centrally. GitHub likewise documents managed settings that can install or block plugins and restrict known marketplaces. Use those controls to enforce the decision, but keep the digest, test evidence, owner, expiration, and revocation reason in your own record.

Failure modes

Confusing format with trust. A valid plugin.json proves that a package can be parsed. It does not prove that its scripts are safe, that its publisher is who it claims to be, or that its requested access is appropriate. Google explicitly lists permission, sandbox, and provenance controls as outside Agent Plugins 1.0, so those checks must be added by the operator.

Static evasion. The Hacker News report on the Cloak and Detonate preprint describes look-alike characters, split commands, and self-extracting payloads placed where scanners may not look. It reports more than 90% evasion against each of eight tested scanners in one dataset of 1,613 malicious skills, while a runtime checker observed operating-system activity. Those are reported research results, not a promise that every scanner fails in the same way. The operational lesson is durable: enumerate hidden files and observe behavior in isolation.

Undeclared runtime dependencies. A skill can appear harmless until a script downloads code, resolves a mutable dependency, or invokes an MCP server. A clean first scan is therefore not a permanent approval. Pin the reviewed revision, deny unlisted network access, and rerun the gate when any bundled component changes.

Permission creep. A task that summarizes a repository should not quietly gain write access to deployment files or broad network access. Compare declared capabilities with observed reads, writes, processes, and destinations. Treat every mismatch as a review event, not as a warning to ignore.

Quality regression hidden by a good demo. NVIDIA reports that its live evaluation measures correctness, discoverability, effectiveness, efficiency, security, tokens, and execution time, and that results vary by product and evaluation design. A single successful demonstration cannot establish a general improvement. Include negative and irrelevant cases and keep a without-skill baseline.

False positives and false negatives. A scanner can flag an innocuous example or miss an obfuscated action. Keep findings explainable, allow a human to review narrowly scoped exceptions, and never let an exception silently widen the sandbox. Conversely, do not fix a suspicious finding by deleting the log; reject or re-run with a documented decision.

Independent component failure. Google notes that a server that fails to start need not prevent a client from loading the plugin’s skills; the client can skip that server and report the failure. That isolation is useful, but it can also surprise an operator whose task assumes the server is available. Add an explicit readiness test for required servers and fail the task when a required dependency is absent.

Stale approval. A registry entry can remain discoverable after ownership, behavior, or policy changes. AWS’s registry model supplies cataloging and approval settings, not a substitute for local revocation. Give approvals an owner and review date, monitor for updates, and make disablement faster than a new install.

FAQ

Is a static scanner enough? No. Use it for structure and known indicators, then run behavioral cases in an isolated environment. The scanner-evasion research reported by The Hacker News is a direct reminder that appearance can be changed while behavior is preserved.

Do I need an Agent Plugin for every skill? No. Google’s explanation says a single skill or a single MCP server can use the simpler component format. Use a plugin when components belong together and need to travel together; apply the same admission gates either way.

How should the with-skill comparison be designed? Hold the prompt, model, task inputs, and grading criteria constant, and change only whether the skill is installed. NVIDIA describes this controlled comparison and runs each condition in its own isolated sandbox. Include a task the skill should handle, a task it should decline or leave unchanged, and an unrelated task.

Should the sandbox have network access? Start with none. Add a narrowly named destination only when the task contract requires it, record the reason, and test that an unlisted destination is denied. A successful answer is not evidence that an unexpected network request is acceptable.

What happens when an MCP server fails? A client may continue loading independent skills and report the server failure, as Google describes. Decide in advance whether the server is optional. If it is required for the task, treat its absence as an error and do not publish a result that implies the full capability was tested.

Where should approval live? Put enforcement in client settings or an approved registry, and keep the evidence and revocation record with the package owner. GitHub documents managed plugin and marketplace settings, while AWS describes authorization and approval settings for discovered resources; neither removes the need for local review.

Reader next step

Choose one low-risk reusable skill and create a staging job with four required artifacts: a manifest-and-digest record, a static-findings report, a sandbox trace, and a with/without task comparison. Start with network denied and synthetic workspace data. Require a reviewer to sign the digest and expiration date before adding the package to a client allowlist. For adjacent controls, use the guide to gate coding-agent pull requests with security scans and the guide to check dependency provenance before merging agent changes . Once the first package passes, keep the same evidence contract for every update and make rejection, rollback, and revocation routine rather than exceptional.