Loading ToolBench
Unified Microsoft MCP Server combining Azure CLI and Microsoft Graph API tools for managing Microsoft 365, Entra ID, and Azure resources
The server defines 24 tools covering Azure CLI and Microsoft Graph operations. While tool names follow verb_noun patterns and schemas are present for all tools, descriptions are generic, parameter documentation is minimal, output schemas are entirely undocumented, and error handling guidance is absent. Most tools fall into the 40-50 range individually due to sparse descriptions and no visible output schema documentation. The codebase shows basic parameter typing but lacks the depth required for production agent usage. Descriptions average ~35-40 characters, below the 194-character production baseline, and do not explain WHEN to use each tool, WHAT it returns, or recovery paths for errors.
Inspect and manage Microsoft Azure resources through the Azure Resource Manager REST API (https://management.azure.com), without the Azure CLI. Use this when the Azure CLI is unavailable or blocked by Conditional Access, or for ARM endpoints that the CLI does not expose. The command is the path after https://management.azure.com, for example /subscriptions/{subscriptionId}/resourceGroups or /subscriptions/{subscriptionId}/providers/Microsoft.Compute/virtualMachines. Methods are GET (read), POST/PUT/PATCH (create or update), DELETE (destroy). Subject to the configured execution policy.
Inspect and manage Microsoft Azure resources by running Azure CLI commands (must begin with 'az'). Use for Azure subscriptions, resource groups, virtual machines, storage accounts, networking, role assignments, and cost data. Examples: 'az account show', 'az group list', 'az vm list -o table'. Subject to the configured execution policy.
Inspect and manage Microsoft 365 and Entra ID (Azure AD) through the Microsoft Graph API (https://graph.microsoft.com). Use this to read or manage users, groups, licenses, mail, calendar, Teams, files, devices, and directory data. GET reads; POST/PUT/PATCH/DELETE write. The command is the path after https://graph.microsoft.com/v1.0, for example /me, /users, /groups, /me/mailFolders/inbox/messages. Subject to the configured execution policy.
No output schemas documented. All 24 tools lack return type documentation. LLMs cannot predict what fields to expect in responses, forcing them to guess at field names for downstream tool chaining.
Descriptions are too brief (avg 28-40 chars vs 194-char baseline). Descriptions do not explain WHEN to use each tool, WHAT it returns, or dependencies. Example: 'List Microsoft Entra ID users' tells LLM nothing about pagination, filtering, or output structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 0 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 29 | - | v1 |
No pagination declared. list_users, list_groups, list_messages, list_events, list_files, list_teams, list_channels, list_devices all accept 'top' parameter but schemas do not document max limits, whether results are capped, or if a next_cursor is returned. Without pagination guidance, LLMs may request thousands of items.
Parameter descriptions are sparse or missing. 'filter' and 'select' appear across 8+ tools but descriptions do not explain OData syntax, supported fields, or format. LLM must guess whether filter='name:john' or filter='name eq john' is correct.
No error handling or recovery guidance in any tool description. Tools do not state what happens on failure, whether errors are retryable, or what the LLM should do next. E.g., 'send_mail' description does not mention rate limits, permission errors, or invalid recipient handling.
No mutation/side-effect warnings. Tools like 'send_mail', 'create_event', 'upload_file', 'send_message' lack explicit descriptions stating they modify state and may have irreversible consequences. LLMs need to know which calls are idempotent and which are not.
Credentials and tokens likely exposed in parameters or response paths. 'get_access_token' returns a token; no indication whether it is stripped from logs or responses before being fed to LLM context. 'execute_azure_cli' accepts arbitrary CLI commands, risk of secret injection.
Parameter naming inconsistencies. Some tools use 'user_id' (get_user) while others use 'user_id or UPN' (get_mailbox, list_messages). Descriptions do not clarify whether both formats are accepted. Tool response fields unknown, LLM cannot reliably pass output from one tool to input of another.
'execute_azure_cli' accepts arbitrary commands post-validation of 'az' prefix, insufficient guard. LLM could pass 'az storage account delete' without confirmation. Destructive tools lack idempotency guarantees or confirmation patterns.
No permission scopes declared. Tools reference Microsoft Graph and Azure CLI but never state what OAuth scopes they require (read:email, write:calendar, etc.). Agents cannot be configured with least-privilege access.