Loading ToolBench
A multi-language MCP server laboratory with Python and TypeScript examples
This is a teaching/demo repository with 9 tools across multiple servers. Critical issues: (1) Parameter descriptions are almost entirely null/missing, 7 of 9 tools have parameters with no descriptions, violating the 100% baseline for A+ tools. (2) Tool naming is inconsistent and generic: 'add' appears twice with different schemas (number vs integer), 'word_count' appears twice identically. No verb-noun clarity for discovery. (3) Output schemas are present in TypeScript code but not visible in Python servers (02_hackernews.py, 03_http.py), making it impossible to verify schema completeness. (4) Duplicate tool names ('add', 'word_count') across the same server create ambiguity for LLMs. (5) No error handling guidance visible in any tool. (6) The 'slow_count' tool claims to demonstrate progress but no progress reporting mechanism is visible in the code snippet.
Add two numbers together.
Add two numbers together.
Echo a message back, optionally repeated.
Look up a single Hacker News story by its numeric ID.
Count to n slowly, reporting progress — demonstrates long-running work.
Fetch the current front page of Hacker News.
Look up the current temperature for a city
Count words, characters, and lines in a block of text.
Parameter descriptions are null/missing for 7 of 9 tools. Rubric baseline: 100% of A+ tools have descriptions for every parameter. Examples: 'add' tool has parameters 'a' and 'b' with null descriptions; 'get_story' has 'story_id' with null description. LLMs cannot infer parameter meaning from names alone.
Duplicate tool names within the same server: 'add' registered twice (number vs integer schema), 'word_count' registered twice identically. This creates ambiguity for LLM tool selection and violates single-responsibility principle. LLMs cannot distinguish between them.
Output schemas not visible for Python servers (02_hackernews.py, 03_http.py). Cannot verify if responses are structured or if they include chaining IDs needed for downstream tool calls. Rubric baseline: 100% of A+ tools have documented return types.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
Count words, characters, and lines in a block of text.
No error handling guidance visible in any tool. Rubric requires: error responses must tell LLM what to do next, categorize errors as retryable/user-fixable/fatal. Example: 'get_story' with invalid story_id should return 'Story not found. Try top_stories() to find valid IDs.' instead of a bare 404.
Generic tool names lack action verbs for clarity. 'add' is clear, but 'echo' and 'slow_count' are vague. Rubric baseline: 90% of A+ tools start with action verbs (get_, create_, search_, list_, update_, delete_). 'slow_count' should be 'count_slowly' or 'count_with_progress'.