Loading ToolBench
A collection of MCP server implementations across multiple labs demonstrating calculator tools, SQL database access, and service integration patterns.
8 tools with basic schemas and descriptions, but significant gaps in parameter documentation and error guidance. All tools have descriptions (10-200 chars, within baseline), and input schemas are present with typed parameters. Error handling is minimal, most tools return JSON errors without recovery guidance. No tool annotations (readOnlyHint, etc.). Naming is clear and verb-forward (query_, list_, describe_). Output schemas are undocumented. This is a typical community workshop server: functional but not production-ready.
Add two numbers together.
Return the last 10 calculations performed in this session.
Show the column names and types for a given table. Use list_tables() first to see available tables.
Divide a by b. Raises ValueError if b is zero.
List all tables available in the sales database.
Multiply two numbers together.
Raise base to the power of exponent.
Parameter descriptions missing for arithmetic tools (add, multiply, power). Parameters 'a' and 'b' lack context, LLMs cannot infer whether they accept integers, floats, or both, or what happens with edge cases (division by zero, negative exponents).
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract structured data. E.g., query_database returns JSON but format (array of objects, field names, types) is undocumented.
Error handling lacks recovery guidance. query_database returns JSON errors like 'Only SELECT queries are allowed' but does not suggest next steps (e.g., 'Try a SELECT query instead').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Execute a read-only SQL SELECT query on the sales database. Returns results as a JSON array. Max 100 rows. Only SELECT and WITH (CTEs) are allowed — no data modification. Tables available: customers, products, orders. Example queries: SELECT * FROM customers WHERE region = 'QLD'; SELECT p.name, SUM(o.total) as revenue FROM orders o JOIN products p ON o.product_id = p.id GROUP BY p.name ORDER BY revenue DESC
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All tools are READ_ONLY but this is not declared in the tool definition, LLMs must infer safety from descriptions alone.
calculator_history has no input schema documented (empty input object). Output schema (last 10 calculations) is not described, LLMs cannot predict structure or plan chaining.