Loading ToolBench
ESPHome MCP server as a Home Assistant custom component. Exposes ESPHome device and entity context from Home Assistant.
ESPHome MCP has well-structured tool definitions with complete schemas and descriptions for all 13 tools. Naming follows verb_noun convention (esp_overview, esp_list_devices, esp_manage_addon). Descriptions are present and substantive (avg ~120 chars). However, several tools lack parameter-level descriptions for complex inputs (esp_manage_addon has 16 params but many lack detail on constraints/dependencies). Output schemas are not documented in the source. Error handling returns structured payloads with error_code and details, but recovery guidance is minimal. Tool composition is sound, each tool has a single responsibility. Security is well-handled: no credentials in params, Supervisor API calls use in-process auth, and responses are normalized to strip sensitive metadata.
Follow ESPHome device builder firmware job progress.
Get ESPHome device builder firmware job status.
List ESPHome device builder devices.
List ESPHome device builder firmware jobs.
Search ESPHome devices known to Home Assistant.
Search ESPHome entities known to Home Assistant.
Manage the ESPHome add-on and call its dashboard API.
esp_manage_addon has 16 parameters with complex interdependencies (action, path, method, body, websocket, options, network, boot, auto_update, watchdog, port, timeout, debug, request_headers, message_limit, message_offset) but lacks detailed descriptions for most. Parameter relationships (e.g., 'websocket=True requires wait_for_close and message_limit') are undocumented, forcing LLMs to guess valid combinations.
Output schemas are not documented in the source code. Tools return structured responses (success, result, error_code, error, details) but the LLM has no visibility into what fields to expect from each tool's result. This forces agents to infer structure from trial-and-error.
Error responses lack recovery guidance. _error() returns {success: False, error_code, error, details} but does not suggest next steps. E.g., 'supervisor_unavailable' should suggest 'This tool requires Home Assistant OS or Supervised installation.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | 2026-07-28+ | v2 |
Return counts and status for the Home Assistant ESPHome integration.
Queue an ESPHome device builder firmware compilation job.
Read ESPHome device builder configuration file.
Run ESPHome device builder stream command.
Search ESPHome device builder YAML files.
Write ESPHome device builder configuration file.
esp_manage_addon's 'action' enum includes lifecycle operations (install, update, rebuild, start, stop, restart, uninstall) but the description does not clarify which are safe to retry (idempotent) vs destructive (uninstall). Agents need this classification to plan safely.
List tools (esp_list_devices, esp_list_entities, esp_list_device_builder_firmware_jobs) accept 'limit' parameters but do not document pagination strategy (offset vs cursor) or whether a 'total' or 'next_cursor' field is returned. Large result sets risk context window exhaustion.