Generic and sparse descriptions. 'Scans inbound text/content for injection patterns and suspicious content' (67 chars) is below baseline (194 chars). LLMs lack context for when to call vs alternatives. Many descriptions under 50 chars.
No output schemas documented. Callers cannot plan downstream tool use or extract required fields. E.g., scanInbound returns a ScanResult but the structure is not visible in tool definitions.
Expand tool descriptions to 100-200 chars each. For scanInbound: 'Detects prompt injection and malicious patterns in user-supplied text before it reaches the LLM. Returns risk score (0 - 100) and matched pattern IDs. Use before processing untrusted input from chat, APIs, or files.'
Document all output schemas explicitly in tool descriptions or as separate schema blocks. E.g., scanInbound returns {action: 'allow'|'block'|'warn', score: number, matchedPatterns: [{id: string, name: string}]}.
Add parameter constraints: for text parameters, specify max length (e.g. 100KB); for path parameters, document allowed root directories and validate no path traversal.
Implement and document input validation. All path parameters must be validated against a whitelist or sandbox. Add error responses with examples: 'Invalid path: /etc/passwd not allowed. Allowed roots: /home, /tmp/agentvet-scan.'
Add actionable error messages. Instead of 'Invalid URL', return 'Invalid URL format. Expected https://example.com. For IP checks, use checkIp instead.'
Rename 'evaluate' to 'evaluatePolicy' or 'evaluateJsonRpc' to clarify intent.
Rename 'scan' to 'scanPath' or 'scanCodebase' to disambiguate from scanFile, scanInbound, scanOutbound.
Expand getRules description: 'Returns the default AgentVet security rules in YAML format. Use this to understand which patterns are checked by scan* tools, or as a template for custom rules.'
Add idempotency markers. Scanning is read-only and safe to retry, but manifest initialization (initManifest) is idempotent, document this via annotations.
Parameter descriptions lack specificity. 'The text content to scan for injection patterns' does not explain format, length limits, encoding, or expected character sets. No enums, ranges, or format constraints visible.
Path traversal risk. Tools accept 'filePath', 'filepath', 'dir', 'skillDir', 'targetPath' without visible validation. No documentation of allowed directories, no symlink guards, no ../ prevention. Code shows 'filePath' acceptance but no sanitization visible in firewall/server.ts.
No error handling guidance. Code shows error responses but no actionable recovery hints. E.g., if checkUrl fails with 'invalid URL', there is no suggestion to use scanFile or checkIp instead.
Vague tool name 'evaluate' (generic action verb). Does not communicate intent, is it evaluating code, policy, risk, or syntax? Ambiguous naming conflicts with pattern rule: 'update_ticket_status' is clear; 'modify_ticket' is vague.
Tool 'scan' is too generic. Name does not indicate what is being scanned or the scan type. Conflicts with multiple more specific scan* tools (scanFile, scanInbound, scanOutbound). LLMs will struggle to choose between them.
Tool 'getRules' has negligible description (20 chars: 'Get the default security rules'). No explanation of rule format, structure, or how to use them. Under hard cap threshold of 20 chars.
No mutual exclusivity or dependency documentation. E.g., 'analyze_file' and 'analyze_directory' could conflict if agent passes both; 'calculateIntegrity' says it needs 'dir' but no explanation of what happens if path is a file not directory.
No rate limiting or timeout configuration visible. External calls to VirusTotal, AbuseIPDB could hang indefinitely. No guidance on retryability or exponential backoff.
checkVirusTotalcheckVirusTotalIpcheckAbuseIPDB
Document which tools require external API keys (VirusTotal, AbuseIPDB) and indicate server-side secret injection pattern in descriptions.
Add batch variants where appropriate. E.g., checkUrlBatch accepting an array of URLs instead of single checkUrl calls in loops.
For file analysis tools, document return structure: 'Returns array of detected permissions with keys: exec (commands run), network (URLs contacted), files (paths accessed), secrets (patterns found), env (environment variables read).'
Add rate limit guidance: 'External threat intel checks (VirusTotal, AbuseIPDB) are rate-limited to 5 calls/minute. Batch multiple checks into one call when possible.'