Loading ToolBench
MCP server for direct AI agent control of lab devices — VISA/SCPI instruments, SSH hosts, REST APIs, serial devices, and Python environments
LabLink defines 5 tools with clear, domain-specific names (connect, disconnect, list_devices, diagnose, system_topology) that follow verb_noun conventions. All tools have descriptions (194 - 280 chars, within baseline 34 - 392 range). Input schemas are present with type declarations and descriptions. However, output schemas are not documented in the source code, the tool docstrings describe what is returned conceptually but do not provide structured JSON Schema for response fields. Error handling is minimal; no recovery guidance or actionable error messages are visible. Tool descriptions are good but could be more concise for LLM parsing. Parameters are well-described but lack enum constraints where applicable (e.g., 'alias' could validate against configured aliases). No tool annotations (readOnlyHint, destructiveHint) are present despite clear risk classifications (WRITE vs READ_ONLY).
Open a session to a device alias and verify communication. Returns device identity, device memory (prior agent notes), documentation pointers, and the device's wiring slice from the system topology.
Run diagnostics for a device alias, or a system audit when alias is omitted. Returns per-alias diagnosis (checks, action items, topology warnings) or system audit (installed drivers, missing system deps, install commands).
Close the session for a device alias.
List all configured device aliases with their type and status ('connected' / 'configured' / 'invalid'). 'configured' means the config parsed, NOT that the device is reachable.
Return the lab's physical wiring map (full topology) or one device's slice. Constraints are advisory context, not enforced. Returns nodes, links, nets, and per-device neighbors and constraints.
Output schemas not documented. Tool descriptions state what is returned (e.g., 'Returns device identity, device memory, documentation pointers') but do not provide structured JSON Schema for response fields. LLMs cannot plan downstream tool calls or extract specific fields without knowing the response structure.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Tools are marked with Risk classifications (WRITE, READ_ONLY) in metadata but these are not exposed as MCP tool annotations. LLMs cannot determine which tools are safe to retry or have side effects.
Minimal error handling and recovery guidance. No visible error classification (retryable vs user-fixable vs fatal) or actionable error messages. E.g., if connect() fails because the device is unreachable, the LLM receives no guidance on whether to retry, call diagnose(), or ask the user.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | <=2025-11-25 | v2 |
Parameter 'alias' lacks enum constraint. The parameter description states 'Device alias matching a configured <alias>.toml file' but does not enumerate valid aliases or provide a way for LLMs to discover them. LLMs must call list_devices() first to learn valid values, adding unnecessary round-trips.
Descriptions are verbose (280+ chars for connect, system_topology). Baseline for A+ tools is 50 - 200 chars. Longer descriptions waste tokens and bury key details. E.g., connect() description could be: 'Open a session to a device. Returns device identity, memory, and wiring topology.'