Loading ToolBench
Secure MCP server for Android device forensic data acquisition via ADB. Supports device connection, shell commands, backups, and data extraction.
Android Forensics ADB Server has significant definition quality gaps. While 15 tools are declared, many lack proper descriptions or have duplicate/conflicting definitions. Naming is inconsistent (e.g., 'get_device_info' appears twice with different signatures at main.py and main.py:10). Descriptions exist but are generic and lack LLM-optimized context (e.g., 'Check if an Android device is connected via ADB' is 45 chars, below the 50-char production baseline). Input schemas are present for most tools but parameter descriptions are sparse or missing. No tool includes error recovery guidance, output schema documentation, or actionable constraint explanations. The server implements command whitelisting (ALLOWED_SHELL_COMMANDS) which shows security awareness, but this is not reflected in tool descriptions. Critical tool chains (e.g., pull_file → analyze_whatsapp) lack documented ID chaining. Tool descriptions do not disambiguate when similar tools are available (list_installed_packages appears twice with different parameter sets). No evidence of pagination support despite forensic operations likely returning large datasets.
Connect to a specific device or verify connection. If device_id is None, uses the first available device.
List all connected Android devices via ADB
Execute a whitelisted shell command on the Android device. Only safe, predefined commands are allowed for security.
Analyze Telegram database. Extracts messages, contacts, channels, and groups.
Analyze WhatsApp msgstore.db database. Extracts messages, contacts, groups, and media references.
Check if ADB is installed and accessible
Check if an Android device is connected via ADB.
Duplicate tool definitions: 'get_device_info' defined twice (main.py lines ~5, ~10 inferred) and 'list_installed_packages' defined twice with incompatible parameter signatures (one uses device_id + system_apps, another uses filter_type enum). LLMs cannot disambiguate and will pick arbitrarily, causing failures.
Descriptions below 50 characters (production baseline p10=34, p90=392): 'Check if an Android device is connected via ADB' (45 chars), 'Get detailed information about the connected Android device' (56 chars, borderline). These lack context about WHEN to call the tool versus similar alternatives (check_adb_status, check_device_connection are near-duplicates with no distinguishing description).
Parameter descriptions missing or generic: 'execute_shell_command' parameter 'command' has description 'Shell command to execute' (27 chars), but does NOT mention the whitelist validation (ALLOWED_SHELL_COMMANDS) visible in main.py code. LLM will attempt to run arbitrary commands, causing failures. 'pull_file' has 'local_name' parameter with no guidance on what happens if file exists (overwrite? error?).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 15 | - | v1 |
Execute an ADB shell command on the device.
Extract system logs from the device.
Get detailed information about the connected Android device.
Get comprehensive device information for forensic documentation
List all installed packages on the device
List installed packages on the Android device.
Pull a file from the Android device.
Update the investigation todo list for planning.
No error recovery guidance: All 15 tools lack actionable error messages. E.g., 'adb_devices' will return {'error': stderr, 'devices': []} on failure, but does not explain what the LLM should do next (check ADB installation? Check USB drivers? Restart daemon?). No tool categorizes errors as retryable vs user-fixable.
Output schemas not documented: None of the 15 tools document their return structure. LLMs cannot know what fields to extract for downstream calls. E.g., 'adb_devices' returns {devices: [serial, state, details]} but 'adb_connect_device' accepts device_id, the connection between these is not documented. Agents cannot plan multi-step chains.
Command whitelist not exposed in tool descriptions: 'adb_shell_command' and 'execute_shell_command' both mention whitelisting in brief descriptions, but do NOT list which commands are allowed. Code shows ALLOWED_SHELL_COMMANDS = {ls, cat, pwd, getprop, dumpsys, pm, am, df, du, ps, top, logcat, id, uname, date, uptime, netstat, ip, ifconfig, settings, content, screencap, wm, find}, LLM has no way to know this and will attempt blacklisted commands, wasting retries.
Parameter constraints missing: Several parameters lack type/format/range guidance visible in code: 'extract_logcat' has 'lines' parameter (integer, default 500) but no min/max documented; 'pull_file' has 'remote_path' parameter with no mention of path traversal restrictions or example valid paths; 'write_todos' has 'todos' array parameter but no schema for array item structure (is it {id, task, status, priority}? What are valid status/priority values?).
No pagination for list operations: 'list_installed_packages', 'adb_devices', and 'extract_logcat' return raw lists with no limit/offset or pagination support. Modern Android devices can have 500+ installed packages and millions of log lines. Returning unbounded lists will exhaust context windows and waste tokens.
Inconsistent parameter naming for device selection: Some tools use 'device_id' parameter (adb_connect_device, get_device_info, list_installed_packages), others use implicit first device or nothing (check_device_connection, extract_logcat, execute_shell_command). No tool documents whether 'device_id' is serial number, index, or other identifier. Agents cannot reliably select which device to target in multi-device scenarios.
write_todos is a state-mutation tool with no description of side effects: Tool description missing (inferred from name alone). No guidance on what 'investigation todo list' is, whether prior todos are replaced/merged/appended, or what format is expected. This tool invites silent data loss if the agent misunderstands the merge behavior.