Loading ToolBench
A collection of example MCP servers demonstrating various patterns: legacy SSE backend, modern HTTP proxy, clothing inventory service, and multi-tool server with resources and prompts
This repository is a collection of tutorial/demo MCP servers with significant quality gaps. Tools lack consistent, LLM-optimized descriptions. Many parameter descriptions are minimal or absent. No tool annotations (readOnlyHint/destructiveHint). Output schemas are not documented. Error handling is minimal. The codebase demonstrates basic FastMCP usage but does not follow production-grade tool patterns. Average tool description is ~50 chars (baseline: 194 chars); many tools have trivial or missing parameter documentation.
Addiere zwei ganze Zahlen
Adds two numbers.
Add or update a clothing item with its price; always returns (item, price)
Provides a legacy data string.
Get the price of a clothing item; always returns (found, price)
Greets a person by name from the legacy SSE system.
List all clothing items with their prices
Multiple tools named 'add' (tools #6, #8, #9) in different servers with inconsistent descriptions and languages. Tool #6 and #9 have no description. Tool #8 has German description 'Addiere zwei ganze Zahlen' with no English fallback. This creates LLM confusion about tool selection and violates the naming clarity pattern.
Tool 'multiply' has no description. LLMs cannot determine when to call this tool or why it exists among others.
Tool 'pizza_salami_price' has a vague, business-specific name that does not start with an action verb. 'get_pizza_salami_price()' or 'get_food_price(item)' would be clearer. Current name does not signal the action; LLMs may not recognize it as a tool for price lookup.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 44 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 36 | - | v1 |
Returns that Pizza Salami at Bella Vista costs 10€.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present in any tool definition. Tool 'add_item' performs a WRITE operation but is not marked with destructiveHint or idempotentHint. Agents cannot distinguish safe from unsafe tools.
Output schemas are not documented for any tool. Tool descriptions do not describe what fields are returned or what structure the response has. Baseline: 100% of A+ tools have documented return types. Example: 'get_price()' returns a Tuple[bool, float] but this is not stated in description or visible return schema.
Parameter descriptions are minimal or absent. Tools like 'search_docs' have a 'query' parameter with description 'Search query for documents' (27 chars, baseline: 72 chars average). No guidance on format, constraints, or examples. LLMs lack actionable parameter details.
No input validation or error handling visible. No try-catch blocks, no validation of parameter ranges/types. If 'get_legacy_data' receives a negative request_id or 'add' receives non-integer input, the response provides no actionable error guidance per pattern:recovery-guide.
Tool 'list_items' returns all items without pagination support. Baseline pattern:paginated-result requires page/offset and limit parameters. Large inventories would return unbounded results, wasting context tokens and inviting LLM reasoning degradation.
Composition issue: 'get_legacy_data' and 'greet_legacy' are from a legacy SSE backend proxied through 'ModernProxyToLegacy'. The proxy server does not clearly document which tools come from which backend, and names do not distinguish proxy-hosted vs. native tools. This violates tool-chain clarity.