LogPilot defines 2 tools with complete JSON schemas and descriptions. Both tools have proper input validation and error handling. However, descriptions are minimal (under 50 chars), parameter descriptions lack detail about constraints and formats, and output schemas are not formally documented. The tools follow verb_noun naming (search, stats) but lack guidance on when to use each. No tool annotations (readOnlyHint, etc.) are present. Error messages are actionable but could be richer. Overall: functional but below production baseline for LLM-optimized tool definitions.
Search log entries by text pattern
Get session statistics
Tool descriptions are too brief (under 50 chars). LLMs need 50-200 char descriptions explaining WHAT, WHEN, and WHY. Current: 'Search log entries by text pattern' lacks context on when to use search vs stats, what format results take, or whether it searches live data or snapshots.
Parameter descriptions lack constraint details. 'severity' param says 'Optional severity filter (ERROR, WARN, INFO, DEBUG)' but does not specify: case sensitivity, whether partial matches work, or what happens if an invalid value is passed. LLMs cannot infer these rules.
Output schemas are not formally documented. Tools return text strings (e.g., 'N matches (Source: live data)\n...') but LLMs do not know the structure. A documented schema would clarify: is the response always text? Are there structured fields? Can results be paginated?
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | <=2025-11-25 | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Both tools are read-only and idempotent, but this is not declared in the schema. Clients cannot infer safety properties without explicit annotations.
Search tool lacks pagination. Results are capped at 50 matches with no offset/limit parameters or next_cursor. Large result sets will be truncated silently, and agents cannot iterate through all matches.