Loading ToolBench
MCP Server for Azure IoT Hub read-only queries
Single tool with basic schema and minimal description. Tool name follows verb_noun pattern (get_iot_device), but description is only 44 chars, below the 50-200 char LLM-optimized baseline. Input schema present with type and description for deviceId parameter. No output schema documented. Error handling returns text responses but lacks actionable recovery guidance. No pagination, no structured output schema, no tool annotations. Composition is minimal (one read-only tool). Overall: functional but significantly below production quality.
Get details and status of an IoT device
Description too short (44 chars). LLM cannot determine when to call this tool vs. alternatives or what context it requires. Baseline: 50-200 chars for LLM-optimized descriptions.
No output schema documented. LLM cannot plan downstream calls or extract required fields (e.g., deviceId, status, connectionState). Baseline: 100% of A+ tools document return types.
Error responses lack recovery guidance. Returns generic text like 'Device doesn't exist or Azure CLI is not authenticated' without suggesting next steps (e.g., 'Try search_devices() with a partial name' or 'Verify AZURE_IOT_HUB_NAME is set').
Credentials exposed in service layer. IotHubService reads AZURE_IOT_HUB_NAME and AZURE_RESOURCE_GROUP from environment but does not validate they are set before use. Missing validation in getIotHubConfig() is not enforced at tool invocation time.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
No input validation or sanitization. deviceId parameter is passed directly to shell command via exec() without escaping. Vulnerable to command injection if deviceId contains shell metacharacters.