Loading ToolBench
Two tools with minimal descriptions (18-20 chars each, below 20-char threshold). Schemas present but parameters lack descriptions. list_windows has empty required array (acceptable for no-param tool); capture_window has window_id with description. No output schema documented. Error handling returns JSON strings but lacks recovery guidance. Tool names follow verb_noun pattern (positive). No parameter validation hints, no pagination, no idempotent markers.
Capture a screenshot of the specified window
List all visible application windows
Tool descriptions are critically short (18-20 chars). 'List all visible application windows' and 'Capture a screenshot of the specified window' lack context on WHEN to use, WHAT is returned, and prerequisites. Rubric baseline: 194 chars average, 10-1024 range required.
No output schema documented. LLMs cannot plan downstream calls or extract fields. list_windows returns JSON array of WindowInfo objects; capture_window returns {path, success} or error JSON. Both need explicit schema declarations in tool metadata.
Error responses lack recovery guidance. FileNotFoundError, CalledProcessError, ValueError return JSON with 'error' and 'install_hint' or 'hint' fields, but do not tell LLM what action to take next (e.g., 'Retry after installing imagemagick' or 'Call list_windows first to verify window exists').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
No parameter validation or constraint hints. window_id parameter lacks format, length, or valid-value guidance. LLMs cannot self-correct invalid IDs without explicit constraints (e.g., 'window_id: string, format: hex or decimal integer, required').
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Both tools are read-only and idempotent, but this is not declared in the tool metadata. Agents cannot infer safety properties without explicit hints.