Loading ToolBench
MCP server for Azure API Management (APIM), Application Insights, and Web App management and diagnostics
26 tools with consistent verb-noun naming (ListApis, GetApiDetails, etc.) and detailed descriptions (avg 180 chars). All tools have input schemas with typed parameters and descriptions. However, output schemas are not formally documented, responses are JSON-serialized objects without declared field types. Error handling returns plain text messages rather than structured error objects with recovery guidance. No tool annotations (readOnlyHint, destructiveHint) despite clear risk classifications. SetActiveResources is WRITE but lacks confirmation/dry-run pattern. Parameter descriptions are good but some lack format constraints (e.g., 'kql' accepts arbitrary KQL without validation hints). No pagination support on list tools despite potential for large result sets.
Given an HTTP method and URL path, finds the matching API and operation in APIM and returns a consolidated explanation: the API name and path, the matched operation, the effective policies (global, API-level and operation-level policy XML), and the backend service destination. Use this to understand the full flow of any endpoint through the gateway.
Returns full details of a specific API in APIM: display name, path, service URL, protocols, revision and description.
Returns aggregated results of availability (ping) tests from Application Insights, showing success rate and average duration per test and location. Use to check if endpoints are up.
Returns full details of a backend service configured in APIM: URL, protocol, TLS settings, credentials and circuit breaker rules.
Returns failed dependency calls (to databases, external APIs, service buses, etc.) tracked by Application Insights. Use to identify which downstream service is causing failures.
Groups and counts exceptions by type and message, showing how many times each error occurred and when it was last seen. Use to prioritize the most impactful exceptions.
Output schemas not formally documented. Tools return JSON-serialized objects but field types, required fields, and nested structures are not declared. LLMs cannot reliably extract data or plan downstream calls.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk classifications. SetActiveResources is marked WRITE but lacks destructiveHint annotation and confirmation pattern.
Error handling returns plain text strings ('APIM error 404: ...', 'Unexpected error: ...') instead of structured error objects with recovery guidance. LLMs cannot determine if errors are retryable or what to do next.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | <=2025-11-25 | v2 |
Returns failed HTTP requests (status code >= 400) from Application Insights. Optionally filters by URL path substring. Useful for identifying which endpoints are failing and for which clients.
Returns the global (service-level) APIM policy XML that applies to all APIs and operations.
Returns recent exceptions from Application Insights, including exception type, message, inner exception and the cloud role that threw it. Useful for diagnosing errors reported by users or monitors.
Aggregates request telemetry by operation name and cloud role, showing total count, failure count, average duration and P95 duration. Use to get a traffic and health summary per endpoint.
Returns the slowest HTTP requests above a given duration threshold from Application Insights. Use to identify performance bottlenecks.
Returns the general site configuration of a Web App: runtime version, Always On, HTTP/2, minimum TLS version, CORS settings, health check path, WebSockets, and logging flags.
Returns the Connection Strings configured for a Web App, including their type (SQLAzure, Custom, etc.). Values are returned as configured in the portal.
Returns details of a specific Azure Web App: runtime stack, state, host names, HTTPS-only setting and client certificate config.
Returns the health check configuration of a Web App, showing the configured health check path and whether it is enabled.
Returns the logging configuration of a Web App: HTTP logging, detailed error logging, and remote debugging status.
Returns the App Settings (environment variables) configured for a Web App. These are key-value pairs injected at runtime.
Lists all operations (endpoints) of a specific API in APIM, including HTTP method, URL template and description.
Lists all APIs registered in the APIM service, including their paths, display names, service URLs and protocols.
Lists all available Azure resources in the subscription that this MCP server can work with: APIM instances (with gateway URL and SKU), Log Analytics workspaces (with workspace ID for App Insights queries), and resource groups. Use the results to call SetActiveResources and configure which ones to use.
Lists all backend services configured in APIM, including their URLs, protocols and circuit breaker rules.
Lists all deployment slots (e.g. staging, canary) configured for a Web App, including their state and host names.
Lists all Azure App Service Web Apps in the configured resource group, including their state, URLs and runtime stack.
Executes an arbitrary KQL query against the Application Insights Log Analytics workspace. Use this for custom diagnostics, ad-hoc investigations or any query not covered by the other tools.
Configures which Azure resources this MCP server should target and saves the selection to %APPDATA%\azure-mcp\settings.json so it persists across sessions. After calling this, all APIM, App Insights and Web App tools will use the specified resources automatically. Use ListAvailableResources first to discover what is available in the subscription.
Shows the currently active resource configuration for this MCP server, including which APIM instance, App Insights workspace and resource group are being used.
List tools (ListApis, ListBackends, ListWebApps, etc.) lack pagination parameters (limit, offset, cursor). Large result sets could exhaust context window or timeout.
QueryLogs accepts arbitrary KQL without validation hints or format constraints. Parameter description lacks guidance on valid KQL syntax, time window limits, or query complexity bounds.