Loading ToolBench
Collection of MCP servers for industrial automation: MQTT with Sparkplug B support, OPC UA client and server
12 tools with mixed quality. Naming is verb-forward and clear (publish_message, subscribe_topic, read_opcua_node). Descriptions are present but brief (avg ~60 chars, below 194-char baseline). Input schemas are well-structured with types and descriptions for most parameters. Critical gaps: no output schemas documented, no error handling guidance, no pagination for list operations, no tool annotations. MQTT tools lack QoS/retain parameter descriptions. OPC UA tools accept opaque node IDs without validation hints. No idempotency or confirmation patterns for destructive ops (publish, write, call_method).
Browse the children of a specific OPC UA node.
Call a method on a specific OPC UA object node.
Get all available variables from the OPC UA server, excluding those under the built-in 'Server' object.
Get MQTT broker connection info and status.
List active MQTT subscriptions.
Publish a message to an MQTT topic.
Read the values of multiple OPC UA nodes in a single request.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., what does publish_message return? Does subscribe_topic return a subscription_id?). Violates pattern:tool and pattern:response-shaper.
Destructive operations (publish_message, write_opcua_node, call_opcua_method) lack confirmation or dry-run patterns. Agents can accidentally publish to production topics or modify critical OPC UA nodes without safeguards. Violates pattern:confirmation-request.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Read the value of a specific OPC UA node.
Subscribe to an MQTT topic pattern.
Unsubscribe from an MQTT topic.
Write values to multiple OPC UA nodes in a single request.
Write a value to a specific OPC UA node.
Descriptions are too brief (avg 50 chars vs 194-char baseline). 'Subscribe to an MQTT topic pattern' lacks context on when to use it vs publish_message, what happens on success, or how to handle errors. Violates pattern:tool-description.
OPC UA node_id parameter accepts opaque strings (e.g., 'ns=2;i=2') with no validation hints or error recovery guidance. If an LLM passes an invalid format, the tool fails silently. Violates pattern:recovery-guide and pattern:param-validation-rules.
list_subscriptions and get_all_variables lack pagination (limit, offset, next_cursor). Large result sets will blow context windows. Violates pattern:paginated-result.