Loading ToolBench
Tool definitions are present but lack rigor. All 4 tools have descriptions (10-100 chars, below baseline 194), but parameter schemas are minimal. Input schemas visible for all tools with basic type info (string, pattern), but lack comprehensive validation, enums, and output schemas. No error handling guidance, no security annotations, no tool composition hints. Naming is clear (verb_noun pattern) but descriptions are too brief to guide LLM selection effectively. No evidence of idempotency, pagination, or response field documentation.
Trigger the Nuki Opener electric strike (buzz-in).
Get the current state and attributes of a Nuki lock.
Lock or unlock a Nuki device.
Execute a full entry sequence: Buzz the main building door and then unlock the apartment door. This demonstrates an agentic workflow for multi-stage entry.
Descriptions are critically short (10-100 chars vs baseline 194). 'Get the current state and attributes of a Nuki lock' lacks WHEN to use, prerequisites, or return value guidance. LLMs cannot reliably select tools with such sparse context.
No output schemas documented. LLMs cannot plan downstream calls or extract required fields (e.g., does get_lock_status return battery_level, is_locked, last_update?). Responses are opaque.
No error handling guidance. Tools lack recovery hints (e.g., 'If lock offline, check WiFi connectivity'). Agents receive raw errors with no actionable next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Destructive tools (set_lock_state, buzz_opener, smart_entry_sequence) lack confirmation or dry-run support. No idempotency guarantees. Agents can accidentally lock/unlock repeatedly or trigger multiple buzzes.
Parameter descriptions are minimal. 'entity_id' lacks format guidance (e.g., 'Home Assistant entity ID format: domain.name, e.g., lock.front_door'). 'action' enum is present but no description of side effects or retry behavior.