Loading ToolBench
Model Context Protocol server for Azure (Resource Manager) — inventory, tags, VM power, and lifecycle with governance controls (access modes, subscription/resource-group scoping, protected groups, location allowlist, delete gating, confirmation, dry-run, audit).
mcp-azure demonstrates strong overall definition quality with consistent patterns across 10 tools. All tools have clear verb-based names (list_, get_, create_, tag_, control_, delete_), detailed descriptions (100-200 chars), and complete input schemas with typed parameters and descriptions. Tool composition is excellent, each tool has a single responsibility, and output references (ARM IDs, resource names) enable tool chaining. Security-focused design with permission gating, confirmation patterns for destructive operations, and audit metadata is evident. However, output schemas are not explicitly documented in the visible code, and error handling guidance could be more specific to individual operations. Descriptions occasionally lack concrete guidance on when to use alternative tools (e.g., get_resource vs list_resources disambiguation). Per-tool scores range from 68 - 78, averaging 72.
Start, stop (deallocate), power off, or restart a virtual machine.
Create a resource group in an approved location.
Permanently delete a single resource by ARM id. Requires admin mode, AZURE_ALLOW_DELETE=true, and confirm equal to the resource name. Provide the api-version for the resource type. Irreversible.
Permanently delete a resource group and everything in it. Requires admin mode, AZURE_ALLOW_DELETE=true, and confirm equal to the resource group name. Irreversible.
Fetch a resource by its full ARM id. Provide the api-version for the resource type.
List the regions available to a subscription.
Output schemas not documented in visible code. LLMs cannot plan downstream tool calls without knowing what fields to expect from list_resources, get_resource, or list_locations responses.
No explicit pagination guidance. list_subscriptions, list_locations, list_resource_groups, and list_resources may return many results but no limit/offset/cursor parameters are visible. Responses could exceed context windows.
Tool disambiguation lacking. Descriptions for list_resources and list_resource_groups do not explain why an LLM should choose one over the other, or when to use get_resource vs list_resources. This forces the LLM to reason through alternatives.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 65 | 2026-07-28+ | v2 |
List resource groups in a subscription (filtered by allowlist).
List resources in a subscription, or within a specific resource group.
List subscriptions the service principal can see (filtered by allowlist).
Merge tags onto any resource by its ARM id (uses the generic Tags API).
Error recovery guidance incomplete. toErrorResult() in server.ts returns error text but does not suggest recovery actions (e.g., 'Try passing subscription explicitly' or 'Check AZURE_SUBSCRIPTION_ID env var'). Errors should guide the LLM toward self-correction.
Tags parameter is generic object{}. No description of tag key/value format, allowed characters, or constraints. LLMs may pass invalid tag structures. Should specify format (e.g., 'object with string keys and string values, max 50 tags per resource').