Skill Security Audit — Scan Before You Install
Skill Security Audit — Scan Before You Install
Runs 100% locally. Never executes the code it audits. Your files never leave your machine.
Third-party skills run with your agent's permissions — they can read files, run commands, and call the network. Most are fine. Some ship with a hard-coded API key, a curl | sh installer, instructions that quietly send your files somewhere, or a script they execute from outside the package. You can't tell from the listing page. Skill Security Audit reads the package the way an auditor would — before you install it.
What it does
Point it at a skill package (a folder containing SKILL.md, or an extracted zip) and it produces a graded audit report:
- Verdict — Pass / Pass with notes / Do Not Install, with the deciding fact stated up front.
- Findings with evidence — every finding cites file, line number, and the verbatim line, plus why it matters and how to fix it.
- By-design behaviors listed separately — a cloud-backed skill should call its API; if that's disclosed and function-required, it's listed as by-design, not dressed up as a risk.
- Bilingual report — English body with a Chinese （中文） executive summary.
How it works (two layers, both inspectable)
- A deterministic scanner (
scripts/scan.py+scripts/rules.json, pure Python 3 standard library — zero dependencies, no network) walks the package and matches a versioned rule set. It produces every hit with file, line, and original text. Evidence never comes from a language model's memory. - A documented judgment standard (
references/audit-standard.md) grades each hit in context: confirmed risk, suspected, by-design, or false positive — with placeholder/example suppression and a stated escalation/downgrade rule for every tier.
The scanner is itself auditable: read the rules, read the code. It only reads the target package; it executes nothing in it, and makes no network calls.
The 10 checks
| ID | Check |
|---|---|
| SEC-01 | Hard-coded credentials (known key formats, credential assignments, private keys) |
| SEC-02 | Destructive commands (recursive deletes, disk operations, permission loosening) |
| NET-01 | Outbound network & data exfiltration (full endpoint inventory, telemetry check) |
| NET-02 | Download-and-execute chains (pipe-to-shell, in-memory download + eval) |
| INJ-01 | Prompt-injection surface (ignore-the-user, concealment, exfiltration instructions) |
| PRM-01 | Over-broad permission claims (wildcard tool grants, sweeping access asks) |
| DEP-01 | Dependencies & opaque payloads (install sources, binaries/archives inventory) |
| OBS-01 | Obfuscated code (decode-and-execute, dynamic execution, embedded payloads) |
| EXT-01 | Execution of files outside the package (unverifiable provenance) |
| DSC-01 | Disclosure consistency (do the "100% local" claims match the evidence?) |
Usage
Install the skill, then ask your agent:
"Audit this skill before I install it:
<path to package>"
The agent runs the scanner, reviews every hit against the audit standard, and writes audit-report.md (scanner output alongside it: audit.json, audit-draft.md). In chat you get the verdict and the findings summary.
Boundaries (stated plainly)
- Static audit only. The audited code is never executed; runtime behavior is not observed.
- Binaries and archives inside a package are inventoried (name, size, SHA-256), not inspected inside.
- Media files (images, audio, video) are not content-reviewed.
- A Pass means no static evidence of risk under the stated checklist version — it is not a security certification, and it says nothing about future versions of the audited skill. Re-audit after updates.
Contents
SKILL.md— the audit workflow the agent followsscripts/scan.py— the deterministic scanner (Python 3, standard library only)scripts/rules.json— the rule set (versioned independently of the scanner)references/audit-standard.md— severity scale, verdict rules, per-check gradingreferences/report-template.md— the fixed report structureexamples/sample-audit-leader.md— a real audit excerpt: this checklist run against one of our own products
Version
Product 1.0.0 · Scanner 1.0.0 · Rule set 1.0.0. Rule-set updates are included with the purchase: as new skill-ecosystem risk patterns appear, they ship as rule additions, and every report records the rule version it was produced under.
