Scan agent skill files for security vulnerabilities. 22 rules for OpenClaw SKILL.md and MCP tool definitions.
Two tools with clear, domain-specific descriptions (security scanning). Both have minimal input schemas (single string parameter each) with descriptions present. Naming follows verb_noun pattern (scan_*). However, output is unstructured text rather than typed JSON objects, limiting downstream composability. No parameter validation constraints (e.g., file existence checks are runtime, not schema-declared). Error handling is basic (returns error strings rather than structured recovery guidance). No tool annotations (readOnlyHint present in metadata but not in schema). Missing pagination for scan_directory despite potentially large result sets.
Scan a directory for agent skill files and check them for security vulnerabilities. Recursively finds OpenClaw SKILL.md files and markdown files with skill frontmatter (name/description fields), then scans each against 22 detection rules.
Scan a single agent skill file for security vulnerabilities. Analyzes an OpenClaw SKILL.md or MCP tool definition file against 22 detection rules covering prompt injection, capability escalation, data exfiltration, encoded payloads, and composition risks.
Output is unstructured text (markdown string) rather than typed JSON objects. Downstream tools cannot parse findings programmatically; LLM must extract severity, rule_id, evidence via text parsing, increasing hallucination risk.
scan_directory lacks pagination support (limit, offset, next_cursor). If directory contains hundreds of skill files, response could exceed context window. No documented result cap.
Input schemas declare only type and description; no format constraints, patterns, or validation rules. Path traversal and symlink attacks are not mitigated at schema level (runtime checks exist but not declared).
Error responses are plain strings ('Error: File not found') rather than structured objects with error codes, recovery hints, or categorization (retryable vs. fatal). LLM cannot determine next action programmatically.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Tool annotations (readOnlyHint, idempotentHint) declared in metadata but not exposed in JSON Schema. LLM cannot infer tool safety properties from schema alone.