Loading ToolBench
Minimal demo of interacting with Azure DevOps via an MCP Server. This is a client application that connects to the @azure-devops/mcp server and demonstrates how to use Azure DevOps tools through an MCP interface with Azure OpenAI.
25 tools with consistent naming (verb_noun pattern) and basic schemas. However, descriptions are generic and lack LLM-optimization guidance. Most parameter descriptions are minimal (10-30 chars), below the 72-char baseline. No output schemas documented. Error handling absent. Tool composition is reasonable but lacks idempotency hints and recovery guidance. Schemas present but sparse on constraints (no enums, ranges, or format specs). This is a typical community server, functional but not production-ready.
List all pipelines in a project
Run a pipeline
List all projects in the Azure DevOps organization
Create a new branch in a repository
Create a comment thread on a pull request
Get a repository by name
Get details of a specific pull request
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields (e.g., work_item_id from create_work_item response). Violates pattern:tool and mxe:include-chaining-ids.
Parameter descriptions are minimal (10-30 chars, well below 72-char baseline). E.g., 'List of fields to retrieve' lacks format guidance. LLMs cannot infer valid field names or constraints. Violates pattern:tool-description.
No error handling guidance. Tools lack recovery hints (e.g., 'If work item not found, try search_work_items'). Agents cannot self-correct on failures. Violates pattern:recovery-guide.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
List all repositories in a project
List pull requests in a repository
Search code in repositories
Search wiki pages
List all teams in a project
Create or update a wiki page
List all wikis in a project
List pages in a wiki
Add an artifact link to a work item
Create a new work item in a project
Get a work item by ID with specified fields
Get work items assigned to the current user
Update multiple work items in batch
Create a link between two work items
Get iteration capacities for a team
Get team capacity for a specific iteration
List all iterations for a team
Update team capacity for an iteration
No enums or constraints on string parameters. E.g., 'link_type' in wit_work_items_link accepts free-form strings ('relates to', 'parent', 'child') but no enum enforces valid values. LLMs hallucinate invalid link types. Violates pattern:constrained-input.
No pagination support visible. Tools like project_list, team_list, repo_list lack limit/offset or cursor parameters. Large result sets risk context window exhaustion. Violates pattern:paginated-result.