Loading ToolBench
Single tool with minimal definition quality. Tool name 'get_sentry_issue' follows verb_noun convention (+), but description is extremely brief (28 chars, below 34-char p10 baseline). Input schema present but parameter description is generic ('Sentry issue ID or URL'). No output schema documented. No error handling guidance, 'Issue not found' and 'Invalid issue ID' are bare strings with no recovery hints. No parameter constraints (enum, format, length). Tool accepts both ID and URL but doesn't explain the parsing logic or expected formats. No mention of prerequisites (SENTRY_AUTH_TOKEN, SENTRY_ORG env vars) in tool description. Response is raw JSON dump with no field documentation.
Get info about a Sentry issue
Tool description is 28 characters, below the 34-char p10 baseline and well below the 194-char average. Lacks WHAT, WHEN, and WHY context for LLM selection.
No output schema documented. LLM cannot predict response structure, field names, or types. Raw JSON dump forces unstructured parsing.
Error responses lack recovery guidance. 'Invalid issue ID' and 'Issue not found' are bare strings with no actionable next steps (e.g., 'Try search_sentry_issues() with a partial name').
Parameter description 'Sentry issue ID or URL' is ambiguous. No format constraints, length limits, or examples of valid inputs (e.g., '12345' vs 'https://sentry.io/organizations/myorg/issues/12345/').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | 2026-07-28+ | v2 |
SENTRY_AUTH_TOKEN and SENTRY_ORG are required environment variables but not mentioned in tool description. LLM has no visibility into prerequisites or failure modes if env vars are missing.