Loading ToolBench
Single tool with minimal definition quality. Tool name 'get_issue' follows verb_noun convention (+), but description is extremely sparse (18 chars, below 20-char threshold). Input schema present with basic type but lacks parameter description entirely. No output schema documented. No error handling guidance. Tool returns raw JSON dump without structure or field documentation. Fails multiple critical checks: parameter descriptions missing, output schema undocumented, error recovery guidance absent.
Get info about a Jira issue
Parameter 'issue_key' lacks description. LLM cannot infer format, constraints, or examples (e.g., 'PROJ-123' vs 'proj-123').
Tool description is 18 characters ('Get info about a Jira issue'). This description lacks WHEN to use it, WHAT it returns, and any prerequisites. Expand to 50-200 chars with actionable context.
No output schema documented. Tool returns raw JSON (cleanFields output) with no field descriptions. LLM cannot plan downstream calls or extract relevant data. Rubric: 100% of A+ tools have documented return types.
No error handling or recovery guidance. If Jira API returns 401 (auth failure), 404 (issue not found), or 500 (server error), tool returns raw error with no actionable next step for LLM. Rubric: error responses must tell LLM what to do next.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 32 | 2026-07-28+ | v2 |
Credentials (JIRA_USERNAME, JIRA_API_KEY) injected via environment variables (correct), but no validation that they exist at startup. Missing credentials cause silent auth failures at runtime. Add startup validation and clear error messages.