Loading ToolBench
Educational repository containing multiple MCP server implementations including a real-tools MCP server with currency conversion and datetime tools, a calculator MCP server with arithmetic operations, and FastAPI-based tool servers.
This server has 5 tools across multiple files with highly inconsistent quality. Two tools (get_current_datetime, convert_currency) in mcp_with_api/server.py are reasonably defined with proper schemas and descriptions. Three tools (add, multiply, subtract) in try/server.py are minimal but present. However, the overall server shows critical gaps: no error handling patterns, no tool annotations (readOnlyHint despite all being READ_ONLY), parameter descriptions are either missing or embedded in docstrings rather than schema, and output schemas are entirely undocumented. The repository structure itself is confusing, it contains multiple unrelated MCP server implementations plus non-MCP FastAPI apps, making it unclear what the canonical server is. No pagination support, no recovery guidance, and descriptions lack depth about WHEN to use each tool.
Add two numbers
Convert amount from one currency to another using live rates. Args: amount: Amount to convert (e.g. 100) from_currency: Source currency code (e.g. 'USD', 'INR', 'EUR') to_currency: Target currency code (e.g. 'INR', 'USD', 'EUR')
Get the current date, time and day of week
Multiply two numbers
Subtract two numbers
No output schemas documented for any tool. LLMs cannot plan downstream operations or extract required fields from responses. The convert_currency tool returns a dict with status/from/to/exchange_rate, but this structure is undocumented.
No tool annotations despite all tools being READ_ONLY. FastMCP supports @mcp.tool(idempotent=True, readOnly=True) but it is not used. LLMs cannot distinguish safe tools from destructive ones.
convert_currency has no error recovery guidance. It catches exceptions and returns {status: error, message: str}, but the message ('No internet connection', 'Currency not found', raw exception) does not guide the LLM on what to do next. E.g., 'Currency not found. Try list_supported_currencies().', but no such discovery tool exists.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 42 | - | v1 |
Parameter descriptions in convert_currency are placed in the tool docstring, not in the @mcp.tool() schema. FastMCP should extract them from type hints and docstrings, but best practice is to ensure they appear in the inputSchema. Verify the generated schema includes all three parameters (amount, from_currency, to_currency) with full descriptions.
Calculator tools (add, multiply, subtract) have trivial descriptions ('Add two numbers', etc.) that do not explain WHEN to use them. They are overly granular, a calculator tool that accepts an expression string or operation list would be more useful to agents than three separate tools.
No validation or error handling in calculator tools. What happens if a, b are out of range? No bounds documented. No guidance if the LLM passes non-integer values (e.g., floats). Responses are plain integers with no metadata.
Repository structure is confusing. Contains mcp_with_api/, try/, mcp_server/, fast_api/, and ollama_fastapi/, multiple independent implementations. No README clarifies which is the canonical MCP server. The scoring assumes the two in-code servers (mcp_with_api/server.py and try/server.py) represent one logical server, but they are separate.