Loading ToolBench
AI-driven mobile app testing via MCP — any AI agent can test any Android app
AgenTest provides well-structured tool definitions with clear naming (verb_noun pattern), comprehensive descriptions (avg 180 chars), and detailed parameter schemas. All 10 tools have input schemas with type definitions and descriptions. However, output schemas are not explicitly documented in the source, and error handling guidance is minimal. Tool composition is strong, each tool has a single responsibility and produces outputs that chain well (e.g., agentest_connect returns @ref tokens used by agentest_run_flow). Descriptions are LLM-optimized and include usage examples. Security considerations are present (no secrets in params). Main gaps: no documented output schemas, limited error recovery guidance, and no tool annotations (readOnlyHint/destructiveHint).
Connect to an Android emulator/device and launch the app. Returns the initial UI screen in compact text format with @ref tokens you can use as selectors. The compact format uses one line per element: @b1 btn "Sign in" — button, ref @b1 @f1 input "Email" — text field, ref @f1 @c1 check "Remember me" — checkbox @s1 scroll — scrollable area @l1 link "Forgot?" — clickable text link @g1 tap — generic clickable (unlabeled) "plain text" — non-interactive text Use refs in subsequent run_flow steps: { "action": "tap", "target": { "ref": "@b1" } } Traditional selectors (id/text/className/description) still work alongside refs. Pass verbose:true to include framework-sync diagnostics.
Get device information including model, Android version, screen dimensions, and installed packages.
Retrieve logcat logs from the device, optionally filtered by package or log level.
Read SharedPreferences data from the app's private storage.
Get a fresh snapshot of the current UI screen. Returns compact text with @ref tokens by default. Use refs in selectors: { "ref": "@b1" }. If a ref is stale (screen changed), you'll get a clear error — just call this tool again for fresh refs. Options: - format: "compact" (default) or "full" (legacy JSON tree with bounds + classNames — use for debugging or layout inspection) - depth: max tree depth (omit for unlimited) - onlyInteractive: true to drop plain text lines (hoisted labels still appear on interactives)
Output schemas not documented. Tool descriptions explain what is returned (e.g., 'compact text with @ref tokens', 'base64-encoded PNG'), but formal output schema definitions are absent from the source. LLMs cannot reliably extract fields or plan downstream calls without documented return types.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Risk levels are declared in the source (READ_ONLY, WRITE, REVERSIBLE, WRITE) but not exposed as MCP tool annotations. Agents cannot automatically infer which tools are safe to retry or have side effects.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
Execute SQL queries against the app's SQLite databases.
Force-stop and relaunch the app. Returns fresh compact tree with @refs. Use between test cases for clean state.
Execute a batch of UI actions and assertions. Stops on first failure. TARGET ELEMENTS using refs from the last tree snapshot: { "action": "tap", "target": { "ref": "@b1" } } Or use traditional selectors (id/text/textContains/className/description/index) — both work. DO NOT add "wait" or "wait_for_stable" steps — the server auto-waits after every action. RESPONSE: includes screenFingerprint and screenChanged. If screenChanged is false and success is true, the UI is exactly where you left it — reuse your prior refs without re-snaphotting. The tree is only included when the screen actually changed or the flow failed. ACTIONS: tap, tap_coordinates, type, clear_text, swipe, swipe_coordinates, long_press, long_press_coordinates, double_tap, double_tap_coordinates, press_key, scroll_to. ASSERTIONS: assert_visible, assert_not_visible, assert_text_equals, assert_text_contains. SELECTORS: ref (fastest — from last snapshot), id (substring), text (exact), textContains (substring), className (short or full name), description (substring), index (0-based Nth match).
Capture a base64-encoded PNG screenshot of the current screen.
Control network connectivity on the device (enable/disable WiFi, cellular, or airplane mode).
Error handling lacks recovery guidance. Descriptions do not explain what to do if a tool fails (e.g., 'If device not found, check adb devices' or 'If query fails, verify database name'). Agents receive no actionable next steps on errors.
Minimal descriptions for simple tools. agentest_screenshot (60 chars) and agentest_device_info (60 chars) lack context on when to use them or what fields are returned. Descriptions should be 50-200 chars with usage hints.
agentest_run_flow 'steps' parameter lacks detailed schema. Described as 'Ordered list of actions and assertions' but no formal schema for step objects (action type, target selector format, assertion fields). LLMs must infer structure from description text alone.