Loading ToolBench
Server has 11 tools with consistent naming (verb_noun pattern) and descriptions present for all tools and parameters. However, descriptions are often too long (200+ chars) and lack actionable constraints. Input schemas are properly typed but lack enums for constrained values (e.g., 'direction' should be enum SHORT|START_LONG|END_LONG, 'key' should enumerate valid key codes). Output schemas are undocumented, tools return free-text strings rather than structured JSON. Error handling is present but generic ('Error: {e}'). No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. Missing pagination for list_apps and list_devices. Parameter descriptions embed examples (e.g., 'com.google.android.youtube.tv') which LLMs may reuse literally.
Disconnect from an Android TV device. Args: device_id: The device identifier.
Scan the local network for Android TV devices using Zeroconf/mDNS. Args: timeout: How long to scan in seconds (default 5). Returns: List of discovered devices with name, host, and port.
Complete pairing by submitting the 6-digit code displayed on the TV. Args: device_id: The device identifier used in start_pairing. code: The 6-digit pairing code shown on the TV screen.
Get the current status of a connected Android TV device. Args: device_id: The device identifier. Returns: Current power state, active app, volume level, and availability.
Launch an app on the Android TV by package name or deep link URL. Use list_apps to see previously discovered package names for a device. Args: device_id: The device identifier. app: The app package name (e.g. "com.google.android.youtube.tv", "com.netflix.ninja", "com.disney.disneyplus") or a deep link URL.
Output schemas undocumented. All tools return free-text strings instead of structured JSON. LLMs cannot parse results reliably or chain tools. E.g., discover_devices returns 'Found devices: - Living Room at 192.168.1.100:6622' but agents need structured {name, host, port} objects.
Constrained parameters lack enums. 'direction' accepts SHORT|START_LONG|END_LONG but is typed as free-form string. 'key' accepts 50+ valid codes but no enum. LLMs hallucinate invalid values. send_key description embeds examples (DPAD_UP, VOLUME_UP) which LLMs may reuse literally in wrong contexts.
No tool annotations despite clear risk levels. remove_device is DESTRUCTIVE but lacks destructiveHint. get_device_status is READ_ONLY but lacks readOnlyHint. Agents cannot distinguish safe from dangerous operations without reading descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
List apps that have been discovered on an Android TV device. Apps are automatically discovered as they are used on the device. The more the device is used, the more complete this list becomes. Args: device_id: The device identifier.
List all saved/configured Android TV devices.
Remove a saved device from the configuration. Args: device_id: The device identifier to remove.
Send a remote control key command to an Android TV. Args: device_id: The device identifier. key: The key to send. Common keys: Navigation: DPAD_UP, DPAD_DOWN, DPAD_LEFT, DPAD_RIGHT, DPAD_CENTER System: HOME, BACK, MENU, POWER Volume: VOLUME_UP, VOLUME_DOWN, MUTE Media: MEDIA_PLAY_PAUSE, MEDIA_STOP, MEDIA_NEXT, MEDIA_PREVIOUS, MEDIA_REWIND, MEDIA_FAST_FORWARD, MEDIA_RECORD Apps: YOUTUBE Info: INFO, GUIDE, SEARCH Numbers: 0-9 Colors: PROG_RED, PROG_GREEN, PROG_YELLOW, PROG_BLUE Channels: CHANNEL_UP, CHANNEL_DOWN Input: DEL direction: SHORT (default), START_LONG (begin hold), or END_LONG (release hold). repeat: Number of times to send the key (default 1). Useful for volume adjustments.
Type text on the Android TV (for search fields, text inputs, etc.). Args: device_id: The device identifier. text: The text to type.
Begin pairing with an Android TV device. The TV will display a 6-digit code. Args: host: IP address of the Android TV (from discover_devices). device_id: A short identifier for this device (e.g. "living_room"). Letters, numbers, underscores only.
Error handling is generic. All tools catch exceptions and return 'Error: {e}' without guidance. E.g., 'Error starting pairing: Connection refused' tells agent nothing about retry strategy or next steps. Missing recovery hints per pattern:recovery-guide.
Parameter descriptions embed example values (com.google.android.youtube.tv, com.netflix.ninja, com.disney.disneyplus in launch_app; living_room in start_pairing). LLMs frequently reuse examples literally, causing failures when agents try to launch 'com.netflix.ninja' on a device that has 'com.netflix.android.tv' installed.