Loading ToolBench
MCP server that provides tools to manage users from a User API backend
This server has 4 tools with basic but incomplete definitions. Tool names follow verb_noun conventions and are clear (getUsers, getUserById, getUsersByName, registerUser). Descriptions exist but are generic and under 50 chars on average, insufficient to guide LLM tool selection. Input schemas are present and use Zod validation, but lack parameter descriptions entirely. No output schema documentation. Error handling is minimal (try-catch logs to console). The server abstracts a REST API but does not guide LLMs through multi-step workflows or provide recovery guidance. This is typical D-grade community work: functional but not production-ready.
Retrieve user by id from the user API
Retrieve all users from the user API
Retrieve users by name from the user API
Register a new user in the user API
All parameter descriptions are missing. Zod schemas define types but LLMs cannot infer intent from Zod objects alone. E.g., getUsersByName accepts 'name' but does not document whether it matches exact name, partial substring, case-insensitive search, or regex patterns.
Tool descriptions are generic and under 50 characters. 'Retrieve all users from the user API' does not distinguish when to call getUsers vs getUsersByName, what the return format is, or whether results are paginated. Baselines show A+ tools average 194 chars with clear context.
No output schema documentation. Tools return JSON-stringified content but LLMs cannot parse the structure (field names, types, pagination). If getUsers returns 1000 users, there is no limit or pagination guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 44 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 34 | - | v1 |
Error handling is non-existent. getUserById catches errors, logs to console, and rethrows, no recovery guidance. If a user ID is not found, the LLM receives an uncaught error with no hint to try search_users or verify the ID format.
registerUser is a destructive operation (WRITE risk) but tool description does not state this explicitly. LLMs cannot distinguish whether this call is safe to retry or has side effects. Confirmation/dry-run pattern absent.
Schema uses bare Zod objects: `{ name: z.string() }` instead of JSON Schema with explicit 'type' and 'description' fields. LLMs expect JSON Schema; Zod is a runtime validation library. This is not a blocking error but is non-standard for MCP.
Input schema for getUsers is empty `{}`. While correct (no parameters), this could be made more explicit in documentation to clarify that getUsers retrieves ALL users and potentially returns large result sets.