Loading ToolBench
Server has 4 tools with basic schemas and descriptions, but significant quality gaps. Tool names follow verb_noun convention (get_*, create_*), which is good. However, descriptions are minimal (10-50 chars), parameter descriptions are sparse or missing, output schemas are undocumented, and error handling is weak. The get_all_authored_pull_requests tool has a critical bug (uninitialized 'result' variable). No tool annotations, no pagination support, no input validation guidance. Schemas use Zod but lack detailed constraints (min/max, patterns). This server would not pass production code review.
Create an Azure DevOps work item (Task or Bug) with provided message and assignee.
Get authored pull requests from Azure DevOps in all repos.
Get all repositories from Azure DevOps.
Get comments on a pull request from Azure DevOps.
Output schemas completely undocumented. Tools return JSON text blobs with no declared structure. LLMs cannot plan downstream calls or extract fields reliably.
Parameter descriptions missing or trivial. 'Repository names' for repos param is too vague. No guidance on format, constraints, or when to use. Violates pattern:tool-description baseline (100% of A+ tools have param descriptions).
Tool descriptions too short (10-50 chars). Baseline is 194 chars. Descriptions lack WHAT/WHEN/WHY context. E.g., 'Get all repositories from Azure DevOps' does not explain when to call it vs other tools or what fields are returned.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
No pagination support. get_all_repos and get_all_authored_pull_requests return unbounded lists with no limit, offset, or next_cursor. Large result sets will exhaust context windows.
Error handling is generic. Errors throw raw messages without recovery guidance. E.g., 'Failed to fetch repositories: ...' does not tell the LLM what to do next or whether to retry.