Loading ToolBench
Three tools with complete Zod schemas and descriptions. query_loki has detailed parameter documentation (8 params, all described, with constraints like limit ≤1000). get_label_values and get_labels are simpler but adequately described. Tool descriptions are functional but brief (10-50 chars); descriptions lack context on WHEN to use each tool or dependencies between them. No output schemas documented, LLMs cannot predict response structure. Error handling present but generic ('Error running Loki query'). No tool annotations (readOnlyHint, etc.). Naming is clear (verb_noun pattern) and follows conventions. Parameters use enums (output format) and validation (limit refine). Missing: output schema documentation, richer error guidance, tool composition hints.
Label name to get values for
Get all available labels
Loki query string
Tool descriptions are too brief (10-50 chars) and lack context on WHEN to use each tool or dependencies. 'Loki query string' and 'Label name to get values for' do not explain the tool's purpose, prerequisites, or relationship to other tools.
No output schemas documented. LLMs cannot predict response structure (e.g., does query_loki return raw text, JSON, or structured objects?). This forces agents to guess and wastes tokens parsing unstructured output.
Error responses are generic ('Error running Loki query') and do not guide recovery. Missing actionable error messages that tell LLMs what to do next (e.g., 'Invalid query syntax. Check Loki LogQL documentation.').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All three tools are read-only but this is not declared in the schema, forcing LLMs to infer safety from descriptions.
query_loki accepts 'from' and 'to' as ISO 8601 strings but converts them to Date objects internally. No documentation of what happens if parsing fails or if timestamps are invalid (e.g., future dates).