Loading ToolBench
an MCP Server that controls io-broker with AI from any MCP Client
The ioBroker MCP server exposes 20 tools with mixed definition quality. Naming is generally verb-forward and clear (getAdapters, setState, createScene), but many tools lack descriptions, parameter descriptions, or visible schemas. Of the 20 tools, 6 have descriptions present (getAdapters, describe, anotate, createState, createScene, describeBulk, deleteItemFromIndex, getAllRooms, getAllObjectsInRoom); 14 lack descriptions entirely. Input schemas are partially visible: tools like getState, setState, createState, getAllObjectsInRoom show structured parameter definitions with types and descriptions, but the majority (getRunningInstances, search, getStatesWithValues, setStateBulk, anotate, setObject, getObject, searchDB, getHistory, setStateBulk, getStateBulk) have no visible schema in the source provided. Output schemas are not documented anywhere. Error handling, validation, and recovery guidance are absent from all tool definitions. No tool declares output structure, pagination, or result limits. The server registers tools via a central allTools.js file using zod schemas for constraints (seen in deleteItemFromIndex.js), but this pattern is not consistently applied. Per-tool scoring averages to 52 due to lack of descriptions, incomplete schemas, and missing error/output documentation.
The anotate tool allows you to attach structured meta information to any ioBroker object (device, channel, or state). This meta information is not used for search, but is returned by the search function and can be used by agents or users to store helpful context, usage notes, quirks, or tips for future reference. Typical use cases: - Remembering which state to use for a specific function (e.g., 'use .ct for color temperature, not .colortemp') - Storing troubleshooting tips or known quirks (e.g., 'sometimes needs to be toggled twice') - Documenting usage examples, preferred settings, or integration notes - Adding tags for logical grouping or categorization (not for search)
Create a new scene object in ioBroker. A scene is a named collection of state/value pairs that can be applied together, for example to set a group of lights, blinds, or other devices to a desired configuration.
Create a new ioBroker state object. Specify id, name, type (default: string), role (default: state), read/write (default: true), and an optional description.
This tool allows you to delete items from the vector index used for semantic search in the ioBroker system.
The describe tool sets a description for an object in the ioBroker system, primarily to enable semantic search. When you use this tool on a state, it will automatically set both common.custom["mcp-server.0"].description and common.custom["mcp-server.0"].enabled to true. Only states with both fields set will be indexed and thus searchable. Describing devices, channels, or folders can serve as a template for their states, but only states with this explicit description and enabled flag are included in the semantic search index.
14 of 20 tools (70%) lack any description. Without descriptions, LLMs cannot determine when to select these tools or understand their purpose.
Most tools (14+) have no visible input schemas in the source code. Parameter types and constraints are undocumented, forcing LLMs to guess valid inputs.
No tool documents its output schema, return type, or structure. LLMs cannot plan downstream calls or extract required fields (e.g., IDs for chaining).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 41 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 35 | - | v1 |
The describeBulk tool sets descriptions for multiple objects in the ioBroker system at once, primarily to enable semantic search. When you use this tool on states, it will automatically set both common.custom["mcp-server.0"].description and common.custom["mcp-server.0"].enabled to true for each state. Only states with both fields set will be indexed and thus searchable. Describing devices, channels, or folders can serve as a template for their states, but only states with this explicit description and enabled flag are included in the semantic search index. Use this tool to efficiently set descriptions for many states in one go.
In IOBroker, an adapter is a software component that extends the functionality of the system, allowing it to interact with various devices and services. This tool retrieves a list of all installed adapters in the IO Broker system including a description, a name, a title, keywords and a link to the adapters readme witch describes the functionality even further. to find divices, states, folders and othe objects you use the patter {name}.0 there might be mutiple instances running you can use the getInstances tool to get a list of all instances of an adapter and if it is currently running or not.
get a list of all valid names of rooms and areas in my iobroker system.
get a list of all valid rooms and areas in my iobroker system
No error handling guidance. Tools do not explain what to do if a call fails, whether to retry, or how to recover. LLMs have no recovery path.
Destructive tools (deleteItemFromIndex, setStateBulk, setState, describe) lack confirmation or dry-run support. No mechanism to prevent accidental data loss.
Tools like getState, setState, and search do not specify pagination, result limits, or how many items may be returned. Large result sets could exhaust context windows.
Parameter descriptions are missing or vague. E.g., 'getHistory options' parameter lacks detail on what 'aggregate', 'start', 'end', 'count' mean or accept.