Plan + track secret rotation across providers from an inventory
Single tool 'tokenrotate_scan' has a basic description (73 chars) but critical gaps in schema documentation and parameter guidance. Input schema visible with 'target' parameter (string type, description present), but output schema is completely undocumented, LLMs cannot plan downstream actions or extract structured data. No error handling guidance, no validation rules, no examples of expected output format. Tool name lacks action verb clarity ('tokenrotate_scan' is vague about what 'scan' returns or when to use it vs other tools). Parameter 'target' description is minimal ('Target path to scan for secret rotation markers') and does not explain expected format, constraints, or what constitutes valid input. No guidance on handling errors (invalid paths, permission denied, timeout). Baseline: 194 chars avg description, 72 chars avg param annotation, this tool falls well short.
Plan + track secret rotation across providers from an inventory. Returns JSON findings.
Output schema completely undocumented. Tool returns JSON findings but LLMs cannot see the structure (fields, types, nesting). Prevents downstream tool chaining and forces LLMs to guess at response shape.
Parameter 'target' lacks validation rules, format constraints, and examples. Description does not explain: Is it a file path? Directory? Glob pattern? What happens if path doesn't exist? What are valid characters?
No error handling guidance. Tool can fail (invalid path, permission denied, timeout) but provides no recovery hints. LLMs cannot self-correct or know whether to retry.
Tool name 'tokenrotate_scan' is vague. Does not start with clear action verb. 'scan' could mean security scan, file scan, or inventory scan. Ambiguous names cause LLMs to conflate tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | 2026-07-28+ | v2 |
No documentation of what 'secret rotation markers' are or how they are detected. LLMs cannot understand when to invoke this tool or what findings mean.