Loading ToolBench
MCP server for controlling and querying SwitchBot devices
The SwitchBot MCP server has 3 tools with visible schemas and descriptions, but significant quality gaps limit its production readiness. All tools have names starting with action verbs (get_, execute_), and input schemas with required fields are present. However, descriptions are generic (10-36 chars), parameters lack detailed constraints and format guidance, and critical output schemas are not documented. Error handling is minimal, tools return raw API errors without recovery guidance. The server does not expose authentication tokens as parameters (correct), but lacks per-tool permission declarations and actionable error messages. No tool annotations (readOnlyHint/destructiveHint) are present despite clear semantic differences (e.g., execute_command is destructive). This server is borderline functional but would benefit from richer descriptions, output documentation, and error guidance before production deployment.
Execute a command on a device
Get SwitchBot device status
Get SwitchBot devices
All tool descriptions are under 20 characters and provide minimal guidance. Tool descriptions should explain WHAT the tool does, WHEN to use it, and any prerequisites for LLM selection. Current descriptions ('Get SwitchBot devices', 'Get SwitchBot device status', 'Execute a command on a device') are too brief to guide agents on when to call each tool or what to expect.
Output schemas are not documented in the MCP tool definitions. The code returns JSON responses (device lists, status objects, command results) but the tool registration does not include an OutputSchema or example output structure. LLMs cannot plan downstream tool chains or extract specific fields without knowing the output shape.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | F | 45 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 43 | - | v1 |
execute_command accepts a free-form JSON string parameter ('command_parameter_json') with no enum constraints, regex pattern, or detailed format guidance. The description says 'Command to send' but does not document the required structure, valid commands, or field names. This invites hallucinated payloads and API errors.
Tool annotations are missing. execute_command is a destructive operation (writes state to physical devices) but has no destructiveHint or idempotentHint annotation. get_switch_bot_devices and get_switch_bot_device_status are read-only but have no readOnlyHint. Tool annotations inform LLM planning about safe retry behavior and confirmation requirements.
Error handling lacks recovery guidance and error classification. The code returns raw API errors or a bare 'Device not found' message. Errors should tell the LLM what to do next (e.g., 'Device not found. Call get_switch_bot_devices() first to list available devices') and categorize as retryable vs. user-fixable vs. fatal.
The execute_command tool has a semantic issue: both 'device_id' and 'command_parameter_json' are required, but no documentation hints that the command structure varies by device type (physical vs. infrared). The implementation distinguishes device types internally, but the LLM has no guidance on which commands apply to which devices.
No permission declarations. Tools should declare required permissions (e.g., 'read:devices', 'execute:command') for least-privilege agent configuration and audit trails. None of the tools specify what OAuth scopes or API permissions they need.
Result pagination is not implemented. If a user has many SwitchBot devices, get_switch_bot_devices returns all of them without limit or pagination. The description says 'Get SwitchBot devices' with no mention of limits or how results are bounded. Large result sets waste context tokens and degrade LLM reasoning.