Loading ToolBench
An autonomous Android automation agent that uses LLM planning and vision-based recovery to execute complex mobile tasks via ADB. Includes Flask REST API for remote triggering from HTTP Shortcuts.
This server has 18 tools with basic schemas and descriptions, but significant quality gaps. Most tool descriptions are under 100 chars and lack context on WHEN to use them or what happens on failure. Parameter descriptions are minimal (e.g., 'Text to type' for type_text). No output schemas documented. No error handling guidance. Tools like tap, scroll, and press_key lack parameter validation rules (e.g., coordinate ranges, valid key names). Memory-based tools (save_memory, delete_memory) have no conflict/overwrite semantics. Device control tools (set_wifi, set_bluetooth) lack state verification or rollback guidance. No tool annotations (readOnlyHint, destructiveHint). The server is not MCP-compliant, it's a custom Flask API with ad-hoc tool definitions, not a proper MCP server.
Deletes a key from persistent memory.json.
Marks the task as complete and exits the agent loop.
Extracts all visible text from the current screen using parsed UI elements. Optionally saves to memory.
Opens an application given its package name with verification and fallback to alternative packages.
Presses a physical or standard Android key.
Saves a key-value pair to persistent memory.json for reuse across steps.
Swipes/scrolls from one coordinate to another on the screen.
Toggle airplane mode on or off.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., what does extract_text return? A string? An array of text blocks?). This violates pattern:tool and pattern:response-shaper.
Parameter descriptions lack validation rules and constraints. E.g., press_key accepts 'key' as a string but does not enumerate valid keys (HOME, BACK, ENTER, etc.) in the description. set_brightness accepts 'level' but does not specify the 0-255 range or percentage format in the description. This forces LLMs to guess valid values.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 33 | <=2025-11-25 | v2 |
Toggle Bluetooth on or off.
Set screen brightness level or toggle auto-brightness.
Toggle the camera flashlight (torch) on or off.
Toggle mobile data on or off.
Set audio volume level for a specific stream.
Toggle WiFi on or off.
Reads screen text, summarizes it via LLM, and optionally saves to memory.
Takes a screenshot of the device and saves it to the screenshots folder.
Taps a UI element at specified coordinates or by element ID/text label.
Types text into the currently focused input field.
No error handling guidance. Tools like open_app may fail (app not found, package name invalid), but descriptions do not explain what the LLM should do next (retry, search for alternatives, ask user). This violates pattern:recovery-guide.
Device control tools (set_wifi, set_bluetooth, set_airplane_mode, set_flashlight, set_mobile_data) lack idempotency guarantees and state verification semantics. If set_wifi is called twice with 'on', does it return success both times? What if the device is already in that state? No documentation.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). LLMs cannot distinguish safe read-only tools (take_screenshot, extract_text) from destructive ones (delete_memory, set_airplane_mode). This violates current MCP spec (2026-07-28) tool annotation requirements.