Phone Farm exposes 11 tools with basic descriptions but significant quality gaps. Most tools lack parameter descriptions (type_text, tap, scroll, launch_app have minimal or missing param docs). Output schemas are undocumented, tools return strings or unstructured text rather than typed objects. Error handling is absent; tools fail silently or return generic error strings. No input validation, no recovery guidance, no pagination support. Naming is reasonable (verb-noun pattern: boot, teardown, tap, screenshot) but descriptions are terse (10-50 chars for most tools, well below the 194-char baseline). The server is functional for basic automation but falls short of production-grade tool design.
Press the Android back button.
Boot an Android emulator, install an APK, and start Appium.
Check logcat for app crashes and ANRs. Returns JSON array of crashes found, or empty array if none.
Check prerequisites: Java, Node, ADB, Appium, disk space.
Launch an app by package name.
Get the current screen's accessibility tree as XML. Returns the UI hierarchy showing all visible elements with their resource-ids, text, bounds, and properties.
Take a screenshot of the current emulator screen.
No output schemas documented. Tools return untyped strings (e.g., screen() returns raw XML, crashes() returns 'JSON array' but no schema). LLMs cannot parse or chain results reliably.
Parameter descriptions missing or minimal. tap() has 3 params (text, resource_id, xy) but no guidance on which to use or mutual exclusivity. type_text() has 3 params with vague descriptions. LLMs cannot determine correct usage.
No error handling or recovery guidance. Tools fail silently (e.g., 'Error: No emulator running' in screen()) or return generic strings. No actionable error messages, no retry hints, no alternatives offered.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Scroll the screen in a direction.
Tap an element on the screen. Provide one of: text, resource_id, or xy coordinates.
Stop the emulator and Appium server.
Type text into an input field.
Descriptions too short (10-65 chars vs 194-char baseline). doctor='Check prerequisites...', back='Press the Android back button.' Lack context for LLM selection and when to use each tool.
No input validation or constraints. scroll() accepts 'direction' as free-form string (should be enum: up|down|left|right). tap() accepts xy as 'x,y' string with no format validation. LLMs will pass invalid values.