Loading ToolBench
MCP server for interacting with MockNotifyService for email and SMS notifications
The server defines 10 tools with basic schemas and descriptions. All tools are properly registered via @mcp.tool() decorator with explicit function signatures. However, there are significant gaps in schema completeness, parameter constraints, and error guidance. Descriptions range from adequate to minimal (10-90 chars), below the production baseline of 194 chars. Most parameters lack type declarations in JSON Schema format, they rely on Python type hints, which are not visible in the MCP protocol layer. No output schemas are documented. Error handling returns JSON strings without actionable recovery guidance. Input validation is minimal. The tools follow a reasonable naming convention (verb-noun pattern) but lack composition for common agent workflows.
Create a new service linked to the user's API key.
Create a message template linked to a service.
Get the delivery status of a previously sent message.
Get all messages for the authenticated user.
Get all services for the authenticated user.
Get all templates for the authenticated user's services.
Check the health status of the MockNotifyService.
No output schemas documented. Tools return JSON strings (untyped text) rather than structured objects. Agents cannot predict response fields or plan downstream tool calls.
Input schemas lack explicit parameter constraints (min/max, enums, patterns). Parameters like 'base_url' accept any string; no validation that it is a valid URL. No regex patterns or length bounds.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 51 | - | v1 |
Send an email notification using a template.
Send an SMS notification using a template.
Register a new user with the MockNotifyService and receive an API key.
Error responses are generic JSON strings ('error' + 'status_code' fields) with no actionable recovery guidance. Agents receive a 400 error with no hint about what went wrong or what to try next. Example: 'Invalid request' does not tell the agent whether to retry, ask the user, or call a different tool.
API key is passed as a tool parameter in every call. Best practice is server-side secret injection via environment variables. Exposing credentials in tool params risks logging them in agent traces and prompt history.
Descriptions are too brief (10-90 chars, below 194-char production baseline). Most lack context about when to use the tool, what happens on success, or prerequisites. Example: 'Get all services for the authenticated user' (52 chars) does not explain that this requires a valid API key or what to do if none exists.
No pagination support. 'get_services', 'get_templates', and 'get_messages' may return large lists unbounded. No limit, offset, or next_cursor parameters. If a user has 500 services, the agent receives all 500, exhausting context window.
No idempotency support or documentation. 'send_email_notification' and 'send_sms_notification' do not return or accept idempotency keys. If an agent retries on a timeout, messages may be sent twice.
No confirmation or dry-run support for destructive operations. 'send_email_notification' and 'send_sms_notification' are irreversible but offer no preview, confirmation prompt, or reversal mechanism.
Parameter descriptions lack format specifications and constraints. Example: 'recipient_phone' says 'Phone number of the recipient (e.g., +1234567890)', the example format is not enforced. What if the agent passes '123-456-7890' or a non-numeric string?
Tools require resource IDs (service_id, template_id, message_id) but do not explain how agents obtain them. Agents must infer that 'get_services' returns service IDs, then use those in 'create_template'. Composition chain is not explicit in descriptions.