Loading ToolBench
AI-powered local network control for TVs and devices. Currently supports Roku TVs via ECP.
The server defines 5 tools for Roku TV control with clear action verbs and reasonable descriptions. However, critical gaps in schema completeness, parameter validation, error handling, and composition patterns prevent a higher score. Tool names follow verb_noun convention (press_key, launch_app, list_apps, get_device_info, power_on) and are appropriately action-oriented. Descriptions are present and moderately detailed (ranging 45-200 chars), meeting the 10-1024 char baseline. However, input schemas lack proper type constraints (e.g., press_key accepts any string for key_name with no enum constraint), parameter relationships are undocumented (launch_app depends on list_apps output), and error messages are generic without recovery guidance. Output schemas are completely undocumented. No pagination support on list_apps despite returning multiple items. No tool annotations (readOnlyHint/destructiveHint/idempotentHint). The implementations contain basic exception handling but do not provide actionable error guidance for LLM recovery.
Retrieves basic device information (model, software version, etc.) as XML.
Launches an application on the TV using its name. Args: app_name: The name of the app to launch (e.g., Netflix, YouTube, Hulu). Case-insensitive. Use list_apps() to see available apps.
Lists all available apps and their corresponding Roku channel IDs. Returns a formatted list of app names and their IDs that can be used with launch_app().
Powers on the TV.
Simulates a single button press on the TV remote. Args: key_name: The name of the key to press (e.g., Home, Select, VolumeUp). Common keys are Home, Back, Select, Up, Down, Left, Right. Special keys are: PowerOn, PowerOff, VolumeUp, VolumeDown, VolumeMute
No enum constraints on string parameters despite fixed known values. press_key accepts key_name as unconstrained string despite only ~20 valid keys (Home, Back, Select, etc.). launch_app accepts app_name as unconstrained string despite APP_MAPPING containing ~25 known apps. This forces LLM to hallucinate valid values.
Output schemas are completely undocumented. All 5 tools return string responses but LLMs have no formal schema describing structure, field names, or format of the response. This violates the 100% baseline for A+ tools.
Error handling provides no recovery guidance. When operations fail (e.g., press_key returns 'Failed to send...'), error messages do not tell LLM what to do next: retry? Call list_apps first? Check TV_IP environment variable? Responses should follow 'recovery-guide' pattern.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 58 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 35 | - | v1 |
No tool annotations. Tools like launch_app and press_key are write operations with side effects but are not marked with destructiveHint or idempotentHint. list_apps and get_device_info are read-only but lack readOnlyHint. This prevents clients from applying appropriate caution.
Undocumented parameter dependencies. launch_app description mentions list_apps() but does not formally document that app_name must be from the list returned by list_apps(). Missing reciprocal reference in list_apps description about which tool consumes its output.
Minimal descriptions on some tools. power_on (22 chars) and get_device_info (61 chars) descriptions are below the median 194 chars baseline and lack context about when to use them, prerequisites, or error cases. Minimum is 10 chars (baseline floor) but descriptions should answer WHAT, WHEN, and what it returns.
get_device_info returns raw XML without transformation. Description says 'as XML' but does not mention this is the raw ECP response which will be verbose and token-wasteful. Should either document expected fields or note that response may be large and suggest parsing to extract relevant fields.
No idempotency declarations or guarantees. Tools like press_key and power_on should be documented as idempotent (calling twice with same input has same effect as once). Without this, LLMs may hesitate to retry on ambiguous failures.