Loading ToolBench
MCP Server for Home Assistant providing tools to interact with Home Assistant entities, services, automations, scenes, scripts, and dashboards
Three tools with basic schemas and descriptions, but significant gaps in parameter documentation and output structure. All tools have descriptions (10-50 chars, below the 50-200 char baseline for LLM optimization). Schemas present but incomplete: 'call_service' and 'list_entities' lack required field specifications. No output schemas documented. Error handling returns plain text rather than structured guidance. Parameters lack type constraints (enums, ranges, formats). No tool annotations (readOnlyHint, destructiveHint). Composition is reasonable (get_state, call_service, list_entities are distinct), but descriptions are too terse to guide LLM selection effectively.
Call a Home Assistant service
Get the state of a Home Assistant entity
List all entities in Home Assistant
Descriptions are too short (10-50 chars vs. 50-200 char baseline). 'Get the state of a Home Assistant entity' lacks WHEN to use it, WHAT it returns, and prerequisites. LLMs cannot reliably select between similar tools with such terse descriptions.
'call_service' has no 'required' constraint on 'entity_id' or 'data', yet the description implies they are optional. Undocumented parameter dependencies (e.g., when is entity_id required vs. optional?) force LLMs to guess, causing silent misuse.
No output schemas documented. Responses are returned as plain text strings (e.g., 'str(result)'), not structured JSON. LLMs cannot parse unstructured text reliably, they must re-parse Python dict representations, wasting tokens and inviting errors.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). 'call_service' is destructive (can turn off lights, lock doors) but carries no warning. LLMs cannot distinguish safe reads from dangerous writes without explicit hints.
Error responses are plain text ('Entity not found', 'Error calling service: ...') with no recovery guidance. Per pattern:recovery-guide, errors should tell the LLM what to do next (e.g., 'Entity not found. Try list_entities() to see available entities.').