Loading ToolBench
A template for creating MCP compatible AI servers in Python
Server has 4 tools with basic definitions but significant quality gaps. The 'add' tool is trivial (arithmetic). Search tools expose internal 'context' dict parameter that should be abstracted server-side. Descriptions are present but generic (10-50 chars for most params). No input validation guidance, no error recovery hints, no output schemas documented. Parameter descriptions lack format/constraint details. Tool names are reasonable (verb_noun pattern) but lack clarity on when to use search_database vs semantic_search. No tool annotations (readOnlyHint, etc.). Overall follows minimal patterns; would not pass production code review.
Add two numbers together.
List all available searchable collections and their schemas.
Search the database using text-based search.
Search the database using semantic vector search.
Internal 'context' dict parameter exposed in search tools. Should be injected server-side via DatabaseContextProvider, not passed by LLM. Violates secret-injection and tool-gateway patterns.
No output schemas documented. LLMs cannot infer what fields search_database or semantic_search return, forcing guesswork on downstream tool chaining. Violates response-shaper pattern.
Parameter descriptions lack format/constraint details. 'limit' has no min/max (e.g. 1-100). 'query' has no length guidance. 'collection' has no enum of valid values. Violates constrained-input pattern.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | <=2025-11-25 | v2 |
No error handling guidance. If search fails or collection not found, LLM has no recovery path. Violates recovery-guide pattern.
Distinction between search_database (text) and semantic_search (vector) unclear in descriptions. LLM cannot determine which to call without trial. Violates tool-description pattern.