TestingBot's official MCP Server for AI-powered testing automation
This server provides 5 tools with reasonable naming and enum-constrained parameters. Tool descriptions are present and mostly adequate (100-250 chars), meeting the 10-1024 char baseline. However, parameter descriptions are sparse or missing entirely on several tools, and output schemas are not documented. The server implements error handling via degraded-mode credential gating (tb_login), but lacks per-tool recovery guidance. Schemas are visible as Zod-style dicts with type constraints, but output structure documentation is absent. No tool annotations (readOnlyHint, destructiveHint) are declared despite clear read/write semantics. The design is reasonably idempotent and uses action verbs (upload, run, get), but lacks explicit per-parameter descriptions for optional parameters.
Get the status of Espresso/XCUITest run(s) for a project. Omit runId to list all runs. Statuses: WAITING, READY (running), DONE, FAILED.
Start an Espresso or XCUITest run on a TestingBot device. Returns immediately with the run ID — it does NOT wait for completion. Poll with getAppAutomateRunStatus, then fetch getAppAutomateRunResults.
Authenticate with TestingBot using device-auth flow (no API key needed)
Upload the app under test for an Espresso (.apk/.aab) or XCUITest (.ipa) run on TestingBot. Returns a projectId for uploadAppAutomateTests and runAppAutomateTest. For Maestro flows use uploadMaestroApp instead.
Upload the test suite for an App Automate project: the instrumented test APK for Espresso, or the zipped XCUITest runner bundle for XCUITest.
Output schemas are not documented for any tool. LLMs cannot infer what fields to expect from response objects, forcing them to guess or make ungrounded assumptions about downstream tool parameter mapping.
Many optional parameters lack descriptions. runAppAutomateTest declares testClasses, skipTestClasses, packages, annotations, testRunner as arrays/strings with minimal or no guidance on format, constraints, or when to use each. This violates the pattern that every parameter needs a description explaining what it controls.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are declared, despite clear semantics: uploadAppAutomateApp and uploadAppAutomateTests are destructive writes; getAppAutomateRunStatus is read-only; tb_login is idempotent. LLMs benefit from explicit hints to plan safely.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 65 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 47 | - | v1 |
Parameter constraints are not documented in descriptions. runAppAutomateTest deviceName supports 'wildcards/regex (e.g. "Pixel 8", "iPhone 1[4-6]", "*")' per description, but no format specification or examples of invalid values. testRunner, locale, timeZone have no constraint guidance.
Error handling lacks recovery guidance. The server implements a degraded-mode gate (returning a credential error for non-tb_login tools when !hasCredentials()), but this is not documented in individual tool descriptions. Tools do not describe what errors they may return or how to recover (e.g., 'If upload fails with 413, file is too large; retry with a smaller app').