Loading ToolBench
ST MCP for CN - Model Context Protocol server for SmartThings home automation integration with Chinese server support
The server defines 6 tools with reasonable naming (verb-based: get_*, execute_*) and mostly present descriptions. Tool annotations are correctly applied. However, schema completeness is inconsistent: get_rooms and get_hub_time lack formal input schemas (empty objects), several parameter descriptions are vague or missing granular validation rules, and output schemas are not documented. Error handling is minimal, no recovery guidance or retryable/fatal classifications. The execute_commands tool is particularly risky: it accepts a destructive 'token' and 'location_id' as parameters (SECURITY RISK), lacks a dry-run or confirmation flow, and provides no per-device success/failure breakdown. Descriptions are moderately detailed but rely on prose examples ('e.g. P1D') rather than formal constraints in some cases.
Execute commands on multiple devices concurrently
Answer questions about past values or trends. Use ISO8601 Duration for `delta_start` and `delta_end` (e.g. P1D for 1 day, PT1H for 1 hour).
Get device status
Retrieve devices based on specified filtering criteria. Parameters: - capability: Optional list of capabilities that devices must have (e.g., ['switch', 'temperatureMeasurement']). - capabilities_mode: Defines how multiple capabilities are matched ('or' returns devices matching any capability, 'and' returns devices matching all specified capabilities). Default is 'or'. - include_restricted: Include restricted devices in the results. Default is False. - room_id: Filter devices by a specific room identifier. - include_status: Include device status information in the response. Default is True. - category: Filter devices by their component category. - connection_type: Filter devices by their connection type (e.g., Wi-Fi, Zigbee).
Get the current time of the hub.
CRITICAL SECURITY: execute_commands accepts 'token' (SmartThings Personal Access Token) and 'location_id' as tool input parameters. Credentials must never appear as tool parameters, they end up in agent trace logs and prompt history, causing credential leakage. Use server-side secret injection via environment variables or vault instead.
get_rooms and get_hub_time declare empty input schemas ({}) but are still registered as tools. These should either require no parameters (which is acceptable) or the schema should be explicitly documented as 'none'. Current state creates ambiguity.
execute_commands lacks error handling guidance, confirmation step, and per-device success/failure reporting. A destructive tool should support dry-run or request user confirmation before executing. If one device fails out of 50, the response should indicate which device(s) failed and which succeeded, not a blanket error.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 52 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 54 | - | v1 |
Get rooms UUID and names
get_device_status description is minimal ('Get device status') and does not explain what 'status' means, what fields are returned, or when to call this vs. other device tools. Descriptions under 50 chars without detail violate the pattern:tool-description rule.
get_hub_time description is vague ('Get the current time of the hub') and does not explain output format, timezone handling, or use case. LLM cannot determine when to call this vs. system time.
Output schemas are not documented for any tool. LLMs need to know what fields to expect so they can plan downstream tool calls. E.g., get_devices returns a dict but the structure is not documented in the server code provided.
get_devices accepts enum constraints for capability and connection_type, which is good. However, parameter descriptions lack validation guidance: What if a user passes an invalid capability? What is the error message? No recovery guidance in descriptions or error handling.
get_device_history uses ISO8601 Duration syntax (P1D, PT1H) but relies entirely on prose examples in the description. The parameter description should include a regex pattern or formal constraint in the schema to guide the LLM and validate input, not just example text.