Loading ToolBench
MCP Server for Azure DevOps Integration by Zubeid Hendricks
Azure DevOps MCP server has basic tool definitions with present but minimal descriptions and incomplete parameter documentation. All 6 tools have names starting with action verbs (list_, create_, get_), which is good. However, descriptions are generic (average ~50 chars, well below the 194-char production baseline), parameter descriptions are missing entirely for most tools, and output schemas are not formally documented. The server lacks error handling guidance, parameter constraints (enums, ranges), and does not follow LLM-optimized description patterns. No input validation is evident in the code. This is a typical community-grade server with functional basics but production-grade gaps.
Create a new Pull Request
Retrieve build definitions for a project
Retrieve team members for a project
Retrieve work items from a project with optional filtering
List all projects in the organization
List repositories in a project
Output schemas are not formally documented. Tools return Dict[str, Any] with no specification of which fields the agent should expect or which fields are safe to pass to downstream tools. This breaks tool chaining and forces LLMs to guess field names.
Tool descriptions are too brief and generic (average ~47 chars vs production baseline 194 chars). E.g. 'List all projects in the organization' lacks context on WHEN to call this, what structure it returns, or prerequisites. Descriptions must answer: what does it do, when should I use it vs similar tools, what does it return?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 43 | - | v1 |
Parameter descriptions are completely missing from the source. The schema shows input types (string) but no descriptions explaining what each parameter means, expected format, or valid values. LLMs cannot infer 'work_item_type' accepts 'Task'|'Bug'|'Feature' from the name alone.
No enum constraints on string parameters that accept a known set of values. 'work_item_type' and 'state' in get_work_items should be enums (e.g. work_item_type: enum=['Task', 'Bug', 'Feature']) to prevent LLM hallucination of invalid values.
Error handling provides no recovery guidance. All methods catch Exception and print to stderr or return empty dicts/lists. Errors like 'Project not found' or 'Invalid branch' do not tell the LLM what to do next (retry, ask user, call a discovery tool). This leaves agents stuck.
create_pull_request is a destructive tool (WRITE risk) with no confirmation or dry-run capability. An agent could accidentally create duplicate PRs or PRs with wrong content. The tool should support a dry-run parameter or require explicit user confirmation before executing.
No pagination support. list_projects and list_repositories return all items with no limit, offset, or page parameters. Large organizations could return thousands of items, blowing the context window. Pagination is missing from the tool interface and likely from implementation.
Tool names use generic operations but lack precision. 'get_work_items' vs 'list_work_items', which should the agent call? The distinction is not clear from names alone. Consistency across the suite would help.
No input validation rules stated in descriptions. E.g. source_branch and target_branch in create_pull_request accept any string, should they validate that branches exist? Descriptions should specify expected format and validation behavior.
Responses include internal metadata (id, url, state) that may not be useful to the chat. No guidance on which fields are suitable to return to the user vs which are for tool chaining. Verbose responses waste tokens.