Loading ToolBench
DeepSeek Harness plugin for Android — build, run, and interact with a live emulator or USB device stream inside a conversation, driven entirely through adb.
Strong tool definitions with comprehensive descriptions and well-structured schemas. All 16 tools have detailed, LLM-optimized descriptions (100-300 chars) explaining WHAT, WHEN, and HOW. Input schemas are complete with typed parameters and descriptions. Tool names follow verb_noun convention (android_devices, android_screenshot, android_interact). However, output schemas are not explicitly documented in the source, and error handling guidance is minimal. No tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications. Parameter descriptions are thorough but some lack explicit constraints (e.g., normalized coordinates 0..1 are explained but not formalized as JSON Schema constraints).
Dump package metadata (`dumpsys package <pkg>`): version, SDK levels, install time, installer, and whether the package is running. A package that is NOT installed is a normal answer (`installed:false` + a note), not an error: the caller asked a question and got a fact.
Dump the stack trace of a running process (`kill -3 <pid>` → `/data/anr/` or logcat crash buffer). The adb shell user (uid 2000) may send the signal only to processes it can signal, and `/data/anr` is `system:system 0770`, so on a production phone BOTH steps usually fail. The tool then degrades to the `crash` log buffer and says so in `engine`, rather than returning an empty stack that reads like "no crash happened". Pass EITHER pid OR packageName (exactly one).
Build an Android app from source (Gradle), install it on a device, and launch it. Detects the project root (gradle.properties or settings.gradle), runs `./gradlew build`, extracts the APK from the build output, installs it with `adb install`, and launches the app. Returns the package name, version, and build output summary. Requires a connected device and a valid Android project.
List all connected Android devices and emulators (`adb devices -l`), enriched with model, Android version, and online/offline state. Includes the machine's available AVDs (emulator images) as a side-channel. Concurrency-safe.
Output schemas not documented. Tool descriptions explain what is returned (e.g., 'Returns a summary (path, byte count, dimensions, device)') but formal JSON Schema output types are not visible in source. LLMs cannot reliably plan downstream tool calls without knowing return field names and types.
No tool annotations despite clear risk classifications. Tools marked READ_ONLY or WRITE lack readOnlyHint/destructiveHint/idempotentHint annotations. LLMs cannot distinguish safe-to-retry tools from irreversible operations without explicit hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2026-07-28+ | v2 |
Search the view tree for nodes matching a text query (case-insensitive substring or regex). Returns matching nodes with their frames in display PIXELS, resource-ids, and content-descs. Use this to locate unlabeled buttons or find a node's resource-id before tapping it with android_tap_element. Concurrency-safe.
Send a touch, scroll, or key event to a device. Touch: tap at (x, y) in normalized 0..1 coordinates (the display is always 1×1 in tool space, regardless of pixel dimensions). Scroll: drag from (x1, y1) to (x2, y2) over duration_ms. Key: send a key code (Back, Home, Recents, etc.). Concurrency-safe.
Launch an already-installed app by resolving its LAUNCHER activity (`monkey -p <pkg> -c android.intent.category.LAUNCHER 1`) — no shell needed. Pass EITHER packageName OR name (exactly one): `name` is a case-insensitive fragment of the package name, resolved against the same listing android_list_apps returns, so several matches come back as a candidate list rather than a guess. Package names of third-party apps CANNOT be guessed; run android_list_apps first when unsure. Use this to OPEN an app; android_build_run is for building and installing one from source. Stable AOSP/GMS packages: Settings com.android.settings, Chrome com.android.chrome, Clock com.google.android.deskclock, Calendar com.google.android.calendar, Camera com.android.camera2, Files com.google.android.documentsui, Play Store com.android.vending.
List the packages INSTALLED on a device (`pm list packages`), enriched with versionName and whether each is a preinstalled system package. Run this BEFORE opening an app: a package name cannot be guessed — a plausible-looking id (a former app name, or the pattern a sibling app uses) is routinely not the installed one. Filter with `query`, a case-insensitive substring matched against the PACKAGE NAME. Android exposes no display label over adb (an app's android:label lives in compiled resources that need aapt2 from the SDK), so a Chinese/Japanese label read off the screen will NEVER match here — match a package fragment, or tap the icon with android_find_text + android_tap_text. User-installed packages only by default; include_system:true adds the preinstalled ones. A listing that FAILS throws with the reason instead of returning an empty list, so count:0 always means the device really has no matching package. Concurrency-safe.
Dump memory usage of a running process (`dumpsys meminfo <pkg>`): TOTAL PSS plus the App Summary categories. The counterpart of ios_sim_leaks's summary mode. Pass EITHER pid OR packageName (exactly one).
List the processes running on a device (`ps -A`): pid and process name. App processes are named after their package (or `<package>:<process>` for a declared sub-process), so this is how you confirm an app is actually running and find the pid other tools want. Filter with a case-insensitive substring. A failed listing throws with the reason rather than returning an empty list.
Capture a screenshot from the frontmost app on a device (`adb exec-out screencap`). Returns a summary (path, byte count, dimensions, device) — never the raw PNG bytes. The image is written to a signed route directory so the web client can fetch it by path. Concurrency-safe.
Tap a view element by its resource-id or content-desc (exact match). Resolves the element's frame from the current view tree, taps at its center in normalized 0..1 coordinates, and optionally verifies the action by checking a counter or text change. Concurrency-safe.
Tap a list/feed row at a RELATIVE position inside row N (fractions of the row frame) instead of guessing absolute screen coordinates, and optionally verify the action by checking a counter change. Real-device safety gate: a tap is planned from a FRESH dump, an out-of-range row index FAILS instead of clamping, an `expect_count` key the row's counters do not contain is refused BEFORE any tap (that is exactly the probe-click failure mode on a real account), and verification reports unverified-with-reason instead of guessing.
Tap a node by matching its text (case-insensitive substring or regex). Resolves the node's frame from the current view tree, taps at its center in normalized 0..1 coordinates, and optionally verifies the action by checking a counter or text change. Concurrency-safe.
Read the visible list/feed rows of the frontmost app as ROWS instead of a raw view tree: each row carries an index, its frame in display PIXELS, the aggregated label (every text and content-desc inside the row subtree), and the counters parsed out of that label — number + classifier, e.g. "57 回复" → 回复=57, in 中文 or English. Use this for RecyclerView / ListView / LazyColumn screens: a list item is a SUBTREE, not an element, and its per-item controls (like, bookmark, share) are usually unlabeled ImageViews with no resource-id — android_tap_element cannot reach them, but android_tap_row taps INSIDE a row at a relative position (see its expect_count). A row is a run of three or more sibling subtrees that share a class and a near-equal height, and at least one of them carries a label; nested runs collapse to the outermost. Counters are parsed heuristically and their keys round-trip, so pass a key EXACTLY as returned to android_tap_row.expect_count. When no rows are found the hint says WHY (the rows are scrolled out of view / the screen exposes no labels at all / this is not a list screen) — never a blanket claim about the app. Rows scrolled off the display are excluded and counted as omittedOffscreen.
Dump the view hierarchy of the frontmost app as a tree (`adb shell uiautomator dump`). Each node carries its class, resource-id, content-desc, text, and frame in display PIXELS. The tree is capped at ~40 KB; nodes scrolled off-screen are included (uiautomator keeps recycled views). Use android_ui_rows for list/feed screens instead of parsing this raw tree. Concurrency-safe.
Minimal error handling guidance. Tool descriptions do not explain what errors can occur, how to recover, or what the LLM should do next. E.g., android_launch_app says 'several matches come back as a candidate list' but does not explain how the LLM should handle or present that list.
Coordinate constraints not formalized in schema. Descriptions state 'normalized 0..1 coordinates' and 'display is always 1×1' but JSON Schema lacks minimum/maximum constraints. LLMs may pass out-of-range values without validation feedback.
Mutually exclusive parameters not formally declared. android_launch_app accepts packageName OR name (exactly one), and android_backtrace accepts pid OR packageName (exactly one), but JSON Schema does not enforce this via oneOf or explicit constraint. LLMs may pass both or neither.