Loading ToolBench
Cross-platform unified accessibility API for AI agents. Exposes Touchpoint UI-automation API as MCP tools so LLM agents can observe and interact with desktop applications.
Touchpoint MCP exposes 26 well-structured UI automation tools with mostly complete schemas and descriptions. Strong naming conventions (verb_noun pattern: click, type_text, press_key, etc.). Most tools include type-annotated parameters and brief descriptions. However, parameter descriptions are often generic or terse (e.g., 'The element ID to click' for click tool), descriptions lack actionable guidance on WHEN to use tools vs alternatives, and error handling is not visible in the tool definitions. Output schemas are not documented. Tool composition is reasonable (separate tools for click, double_click, right_click rather than a single mode-based tool), but descriptions could better explain disambiguation and dependencies (e.g., when to use click vs click_at, or when screenshot is needed before element queries). The alias system is mentioned in server.py but not exposed to the LLM in tool descriptions, potentially causing confusion about element ID formats. No visible rate limiting, permission gates, or audit trail declarations. Protocol is STDIO-only, which caps the protocol readiness score to 50 maximum.
List applications that expose accessibility elements. Queries the backend for all applications currently registered in the accessibility tree.
Click an element by ID.
Left-click at screen coordinates.
Configure runtime options for Touchpoint.
Return diagnostics information about the backend and available targets.
Capture a diff between the current window state and a baseline snapshot. Use in no-vision mode to verify action outcomes.
Double-click an element by ID.
Parameter descriptions are minimal (30-50 chars) and lack actionable guidance on usage, dependencies, or constraints. E.g., 'The element ID to click' does not explain how element IDs are formatted, where they come from, or what happens on failure.
No output schemas documented for any tool. The agent cannot know what fields to expect in responses (e.g., elements() returns a list, but what fields does each element have? Is element_id guaranteed?). This forces agents to guess and slows downstream tool composition.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 65 | 2026-07-28+ | v2 |
Double-click at screen coordinates.
Get UI elements from the accessibility tree. Scoping parameters (app, window_id, tree) are passed to the backend. Filtering parameters (states, role, named_only, filter) are applied afterwards on the returned elements.
Find UI elements by query text. Matches text in 4 stages: exact -> contains -> word match -> fuzzy.
Find the best matching window for a screenshot crop or UI action.
Get a single element by ID. Returns detailed element information.
Press a keyboard combination. All keys are held down in order, then released in reverse.
Return the number of physical monitors detected.
Move the mouse pointer to screen coordinates.
Move a window to new screen coordinates.
Press and release a single key.
Resize a window to new dimensions.
Right-click an element by ID.
Right-click at screen coordinates.
Capture screen pixels and return a PIL.Image.Image. With no arguments, captures the entire virtual desktop (all monitors). Use the optional parameters to crop to a specific region. Only one of app, window_id, element, or monitor may be specified.
Scroll at a screen position.
Set the value of a text-input or numeric element.
Capture a structured tree of the active window with element IDs. Use when find() returns multiple elements with the same name and you need to understand container structure, or when you need no-vision verification.
Type a string into the currently focused widget. Special characters are converted to keystrokes: \n — Enter (line break), \t — Tab (move to next field), \b — Backspace (delete previous character).
List all windows from the accessibility tree. Returns every window the backend can see — visible, hidden, active, inactive. No filtering is applied.
No error recovery guidance in tool descriptions. Tools like click, set_value, and screenshot do not document what happens if the element does not exist, is not interactive, or the screenshot fails. Agents have no guidance on retrying or calling alternative tools.
Tool descriptions do not explain the alias system (e.g., 'atspi1', 'cdp3'). The server assigns short ephemeral aliases to reduce token usage, but agents are not told this is happening. They may become confused if long IDs appear in logs or errors.
Enumerated parameters lack formal enum constraints in schemas. E.g., direction in scroll is documented as 'One of up, down, left, right' but is likely a free-form string in the schema. Agents will hallucinate invalid values.
No visibility into permission gates, scope declarations, or audit logging. The tool definitions do not state what permissions are required (e.g., 'requires write:accessibility' for click), making it impossible to assess least-privilege configurations.
Mutually exclusive parameters not declared. E.g., elements() accepts 'app', 'window_id', 'tree', 'root_element', and 'source', the relationships and exclusions are not documented in parameter descriptions. LLMs will pass conflicting values.
No pagination or result limits documented for discovery tools (apps, windows, elements, find). If elements() can return thousands of results, the response may blow the context window. Limits should be enforced and stated in descriptions.
click, double_click, right_click, click_at, double_click_at, right_click_at tools all do similar things with slight variations. Tool descriptions do not explain when to use click vs click_at or how to choose between them. This forces agents to reason about overlapping functionality.