Loading ToolBench
MCP server for editing Android preferences
Pref-Editor has well-structured tool definitions with complete schemas and consistent descriptions. All 7 tools follow a clear verb_noun naming pattern ('list_*', 'read_*', 'change_*', 'delete_*', 'add_*'), which is excellent. All parameters have type definitions and descriptions. However, descriptions are formulaic and lack context about prerequisites, when to use each tool, or what the LLM should expect. For example, 'change_preference' lacks guidance on failure modes or whether the value undergoes type coercion. Parameter descriptions are minimal (e.g., 'The device's serial number' repeated identically across all tools). Output schemas are not explicitly documented, the code returns text/JSON content but doesn't describe what fields to expect in the preference objects. Error handling is present but generic ('Unknown error'). Security considerations around device access are not documented. The tool set is well-composed (each tool has one clear responsibility) and parameter naming is consistent across tools.
Adds a new preference given the name, value and type.
Changes the value of an existing preference
Delete an existing preference
Lists connected Android devices
Lists apps installed on device
Lists preference files for an app
Reads all user preferences in a file
Parameter descriptions lack actionable detail. All deviceId parameters repeat 'The device's serial number' without explaining how to obtain a device ID, what format is expected (ADB serial?), or what happens if the device is not connected. LLMs cannot distinguish between 'connected device serial', 'IP:port', or 'model number'.
Tool descriptions lack context about when to use each tool and prerequisites. For example, 'read_preferences' should note 'Call list_files first to discover valid filenames.' 'change_preference' should clarify whether it creates the preference if missing or errors. 'add_preference' should explain what happens if the preference already exists.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 60 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Output schema not documented. Code returns content arrays with type 'text' or JSON, but tool documentation does not specify what fields the JSON contains or how to parse the response. For 'read_preferences', the response is an array of JSON objects but the schema of each object is not specified. LLMs cannot reason about downstream chaining without knowing output field names.
Error handling is generic. All tools return 'Unknown error' or a raw error message without actionable recovery guidance. If a device is not found, the error should suggest 'Call devices() to see available devices' instead of a bare stack trace. If type validation fails in add_preference, the error should list valid types (integer, boolean, float, double, long, string) that were accepted.
add_preference's 'type' parameter accepts free-form strings ('integer', 'boolean', etc.) but should be an enum. The description lists valid types but the schema doesn't enforce them. LLMs may pass invalid values like 'int' or 'bool' instead of the expected 'integer'/'boolean'.
Destructive tools (delete_preference, change_preference) lack confirmation/dry-run capability. An agent could accidentally delete all preferences or overwrite critical settings without a chance to review. Consider a 'dry_run' parameter or a separate 'confirm_delete_preference' tool.
Security permissions not declared. The tools interact with Android devices and can read/write private app preferences, but no scope declarations (e.g., 'device:read', 'preferences:write') are present. This prevents least-privilege agent configuration and audit trails.