Playwright + TypeScript E2E and visual regression suite for Automation Exercise, built with an agentic Claude Code workflow
This server exposes 8 API client tools with minimal schema documentation and generic descriptions. Most tools lack parameter type definitions beyond basic strings and objects. Descriptions are present but generic (avg ~50 chars, well below the 194-char baseline). No input validation guidance, no error recovery hints, and no output schemas documented. Tool names follow verb_noun convention (good), but parameter descriptions are sparse. The 'account' parameter in createAccount and updateAccount is an untyped object with only a comma-separated list of fields in the description, no JSON Schema structure. No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. This is a typical community-grade server with functional but poorly-documented tools.
Creates a new account via the account API endpoint
Deletes an account via the account API endpoint
Retrieves the list of all brands from the catalog API endpoint
Retrieves the list of all products from the catalog API endpoint
Retrieves user details by email address via the account API endpoint
Searches for products by term via the catalog API endpoint
Updates an existing account via the account API endpoint
Verifies login credentials via the account API endpoint
Untyped object parameters lack JSON Schema structure. createAccount and updateAccount accept an 'account' object with only a comma-separated field list in the description, no formal schema. LLMs cannot validate structure or infer required vs optional fields.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., does getProductsList return product_id, name, price?). Agents must guess structure.
No tool annotations despite clear risk levels. deleteAccount is DESTRUCTIVE but lacks destructiveHint. verifyLogin and getUserDetailByEmail are READ_ONLY but lack readOnlyHint. Agents cannot reason about side effects.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | 2026-07-28+ | v2 |
Generic descriptions lack context for tool selection. 'Retrieves the list of all products from the catalog API endpoint' (48 chars) does not explain when to call this vs searchProduct, what fields are returned, or pagination behavior.
No error handling guidance. If deleteAccount fails (e.g., wrong password), no recovery hint is provided. If getUserDetailByEmail returns 404, no suggestion for alternative lookup methods.