Loading ToolBench
MCP Server for controlling smart home devices from multiple brands
Server has 25 tools with basic descriptions and schemas, but significant gaps limit production readiness. All tools have descriptions (10-150 chars, mostly adequate), and input schemas are present with type definitions. However, parameter descriptions are minimal or missing context about constraints, enums, and valid ranges. Output schemas are undocumented, responses are JSON strings without declared structure. Error handling is generic ('Error: {e}') with no recovery guidance. Tool naming is verb-first and clear, but composition issues exist: midea_* and virtual_* tools duplicate functionality across brands without clear differentiation. No tool annotations (readOnlyHint, destructiveHint) despite clear read/write semantics. Generic tools (list_devices, control_device) lack parameter constraints (e.g., no enum for 'type' or 'capability'). Baseline: 549 production tools average 194 chars for descriptions; this server averages ~80 chars, often too terse.
Control any device by setting a capability value (e.g. power, temperature, brightness, mode).
Get the current state of any device by its ID.
Activate a scene by name. Executes all actions defined in the scene.
List all rooms in the home and their devices.
List all available scenes that can be activated.
List all known devices across all brands. Optionally filter by brand or device type.
Discover Midea AC devices on the local network. Requires MIDEA_ACCOUNT and MIDEA_PASSWORD configured in .env. Results are cached locally for subsequent commands.
Output schemas undocumented. Tools return JSON strings without declared structure. LLMs cannot plan downstream calls or extract fields reliably. E.g., control_device returns {device_id, capability, before, after, state} but this is not documented in tool definition.
Parameter constraints missing. 'capability' in control_device accepts any string with no enum. 'mode' in midea_set_mode and virtual_set_mode should declare enum [cool, heat, auto, dry, fan] in schema, not just description. 'speed' should be enum [auto, low, medium, high, full]. LLMs hallucinate invalid values without formal constraints.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | <=2025-11-25 | v2 |
Get the current status of the Midea air conditioner. Returns: power state, mode, temperatures, fan speed, and feature states.
Enable or disable eco/energy-saving mode on the Midea air conditioner.
Set the fan speed of the Midea air conditioner. One of "auto", "low", "medium", "high", "full".
Set the operating mode of the Midea air conditioner. One of "cool", "heat", "auto", "dry", "fan".
Control swing/oscillation of the Midea air conditioner.
Set the target temperature of the Midea air conditioner (16-31°C).
Enable or disable turbo/powerful mode on the Midea air conditioner.
Turn off the Midea air conditioner.
Turn on the Midea air conditioner.
Get current state of a virtual device.
List all virtual devices and their types/capabilities.
Set brightness of a virtual light (0-100).
Set fan speed of a virtual AC device. One of "auto", "low", "medium", "high", "full".
Set mode of a virtual AC device. One of "cool", "heat", "auto", "dry", "fan".
Set position of a virtual curtain (0=closed, 100=open).
Set temperature of a virtual AC device (16-31°C).
Turn off a virtual device.
Turn on a virtual device.
Duplicate tool sets across brands (midea_* vs virtual_*) without clear composition guidance. LLMs cannot distinguish when to use midea_turn_on vs virtual_turn_on. No tool description explains which brand to target or how to select between them. Violates single-responsibility principle.
Error handling is generic and non-actionable. All errors return 'Error: {e}' with no recovery guidance. E.g., 'Device not found' should suggest 'Run list_devices to see available devices' (which it does in some cases, but inconsistently). No error categorization (retryable vs fatal).
No tool annotations. Tools with clear read/write semantics (e.g., midea_turn_on is destructive, list_devices is read-only) lack readOnlyHint and destructiveHint annotations. Agents cannot reason about safety or idempotency without explicit hints.