Loading ToolBench
MCP server for interacting with Azure DevOps pull requests, enabling retrieval of PR details, threads, and creation of comment threads
Three tools with basic schemas and descriptions, but significant gaps in parameter documentation, output schema clarity, and error handling. Tool names follow verb_noun convention (GetPrDetails, GetPrThreads, CreatePrThread), which is good. However, descriptions are minimal (34-65 chars, below the 50-200 char LLM-optimized baseline). Input schemas are present but incomplete: IncludeStatuses and ExcludeStatuses lack type constraints (should be enums or have format guidance); Context object in CreatePrThread has optional fields with unclear semantics (StartOffset/EndOffset descriptions are confusing). Output schemas are documented in model classes but not integrated into tool definitions. No error handling guidance, no recovery paths, no validation rules in descriptions. No tool annotations (readOnlyHint, destructiveHint). Composition is reasonable (three separate concerns), but parameter naming could be more explicit (e.g., 'threadStatuses' instead of 'IncludeStatuses').
Create a comment thread on a specific location in code
Get pull request details from Azure DevOps
Get pull request threads from Azure DevOps
Minimal tool descriptions (34-65 chars). LLM-optimized descriptions should be 50-200 chars and answer WHAT, WHEN, and WHAT IT RETURNS. Current descriptions lack context for tool selection and downstream planning.
Status filter parameters (IncludeStatuses, ExcludeStatuses) lack enum constraints or format guidance. Descriptions mention examples ('active', 'closed', 'pending') but do not declare valid values formally. LLMs will hallucinate invalid statuses.
CreatePrThread Context object has confusing optional fields (StartOffset, EndOffset). Descriptions say 'character offset' but do not clarify: are both required together? What if only StartLine is set? Undocumented dependencies cause silent misuse.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 38 | <=2025-11-25 | v2 |
No error handling guidance. Tools do not document what happens on invalid PR URL, missing file path, or API failures. LLM receives raw errors with no recovery path (e.g., 'try search_pr_by_title() first').
No tool annotations. CreatePrThread is destructive (writes to PR) but lacks destructiveHint. GetPrDetails and GetPrThreads are read-only but lack readOnlyHint. Agents cannot infer retry safety or side effects.