Loading ToolBench
MCP server component of agentic-patterns: core agentic AI patterns as small, runnable, tested examples
26 tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. Most tools have 1-3 required parameters with minimal constraint information. Descriptions average ~40-80 chars, below the 194-char baseline for A+ tools. Error handling is generic ('ERROR: {exc}' string wrapping). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk classifications. Duplicate tool names (lookup_order appears twice, tools 1 and 24). Security-sensitive tools (delete_account, send_refund, cancel_subscription) lack confirmation/dry-run patterns.
Book a hotel room for a stay. Fails if the city has no availability.
Cancel a customer's subscription, effective immediately.
Confirm a biometric identification match against a candidate record.
Convert an amount between currencies using fixed exchange rates.
Permanently delete a user account.
Combine a weather forecast and an attraction list into a short itinerary.
Estimate total hotel cost for a stay, without booking it.
Parameter descriptions missing from all schemas. Rubric requires every parameter to have a description explaining what it controls. Currently only parameter names and types are visible; LLMs cannot infer meaning from bare names like 'status', 'type', 'reason'.
No output schemas documented. Rubric requires documenting what fields the LLM should expect in responses. Without this, agents cannot plan downstream tool calls or extract the right data.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
Look up the next stock exchange holiday for a country.
Look up the latest stock price for a ticker symbol.
Get the weather forecast for a city.
Issue a refund for an order.
Look up a customer's loyalty tier. Read-only, no side effect.
Look up an order's status by id.
Look up an order's shipping status.
Delete a memory by key.
Search stored memory by similarity to a query.
Persist a fact for later sessions, under a short key.
Overwrite an existing memory by key.
Send a password reset link to the email on file for an account.
Reverse the most recent refund sent to a customer.
Schedule a calendar meeting on a given date.
List top attractions in a city.
Search internal help documentation for a query.
Send a refund payment to a customer. Irreversible once sent.
Track a package shipment by tracking number.
Translate text into a target language.
Destructive/irreversible tools lack confirmation or dry-run patterns. delete_account, send_refund, cancel_subscription, and reverse_refund can cause data loss or financial impact but have no confirmation step. Rubric requires 'confirm_before_execute' pattern for irreversible operations.
Duplicate tool name 'lookup_order' (tools 1 and 24). LLM will be confused about which to call. Consolidate into a single tool or rename one to clarify the distinction (e.g., lookup_order_status vs lookup_order_shipping).
Generic error handling. ToolRegistry.execute() wraps all exceptions as 'ERROR: {exc}' strings. Rubric requires error responses to tell the LLM what to do next and categorize errors as retryable, user-fixable, or fatal. Current approach gives agents no recovery guidance.