Loading ToolBench
Single tool with basic schema and description. Tool name 'get_star_count' follows verb_noun convention (+). Description is present but minimal (47 chars, below 50-200 char baseline for LLM optimization). Input schema has proper types and descriptions for both parameters (+). However, output schema is undocumented, LLMs cannot plan downstream calls or know what fields to expect. No error handling guidance; bare 'None' return on failure gives agent no recovery path. No pagination, no rate-limit guidance, no security annotations. Tool is read-only but lacks explicit readOnlyHint annotation.
Fetch star count from a GitHub repository.
Output schema undocumented. Tool returns int or None, but LLMs need explicit field documentation to plan chaining and extract results reliably.
No error handling guidance. Function returns None on failure with only stderr logging. LLM receives no actionable recovery hint (e.g., 'Invalid repo, try search_repos()' or 'Rate limited, retry in 60s').
Tool description too short (47 chars). Baseline for LLM-optimized descriptions is 50 - 200 chars. Current description lacks context on when to use this tool vs alternatives, prerequisites (GitHub token), or what the return value represents.
No tool annotations. Tool is read-only but lacks readOnlyHint annotation. Agents cannot infer safety/idempotency from metadata.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Bare exception handling. Catch-all 'except:' clause swallows all errors (network, auth, parsing, rate-limit). No differentiation between retryable (network timeout) and user-fixable (invalid repo) errors.