Loading ToolBench
MCP server for user management operations
Single tool with partially defined schema and minimal parameter documentation. Tool name 'create-user' follows verb_noun convention and is clear. Description is adequate (47 chars, within bounds). However, critical gaps: (1) no parameter descriptions visible in schema registration; (2) input schema lacks type constraints (all params are bare z.string() with no min/max length, format hints, or validation rules); (3) output schema is unstructured (returns generic text content instead of typed object fields); (4) no error handling guidance, generic catch returns 'failed to save user' with no recovery hints; (5) 'address' parameter is accepted but never used in createUser function, suggesting incomplete implementation; (6) STDIO transport caps protocol readiness at 50, but definition quality itself reflects schema and documentation deficits. Per Arcade baselines, 100% of A+ tools have documented return types and parameter descriptions, this tool has neither.
Create a new user in the database
Missing parameter descriptions in tool schema. All four parameters (name, email, address, phone) lack LLM-guidance text explaining what each controls, valid format, or constraints.
Input schema uses unconstrained z.string() for all parameters. No min/max length, email format validation, or pattern enforcement. Allows LLM to pass arbitrary strings (e.g., empty string, 10000 chars) that will fail at runtime.
Output schema is unstructured text ('User{id} created successfully'). No documented return type with typed fields. LLM cannot extract the created user_id for downstream calls. Per Arcade baseline, 100% of A+ tools have documented return types.
Generic error handling with no recovery guidance. Catch block returns 'failed to save user', does not tell LLM what to do next, whether the error is retryable, or what the underlying cause might be.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 52 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 33 | - | v1 |
'address' parameter is accepted in tool schema but never passed to createUser() function. Dead parameter suggests incomplete implementation and will confuse LLMs into passing unused data.
No input validation. Tool accepts any string (empty, null, XSS payloads, SQL injection). Per Arcade pattern:tool-gateway, all agent input must be sanitized and validated before use.
Tool modifies state (writes to JSON file) but error handling does not distinguish retry-safe failures from permanent ones. Bare catch may mask file permission issues, disk full, or JSON corruption, agent cannot recover intelligently.