Loading ToolBench
A study project demonstrating MCP server implementations with multiple transport mechanisms (SSE, FastMCP) and integration with a FastAPI backend for arithmetic operations and CSV data querying
This server exhibits severe quality gaps across all dimensions. Tool names lack action verbs (query, get_columns are weak; calculate_* and *_post are generic). Descriptions are minimal or generic ('Add two numbers together' appears 4 times verbatim). Input schemas exist but lack parameter descriptions, the 'query' parameter in the query tool has only a type, no guidance on valid syntax or constraints. Output schemas are completely undocumented. Error handling is minimal (generic try/catch blocks returning error strings). The server mixes concerns: csv_mcp_server.py handles CSV queries, remote_server.py proxies to an external API, and api_server.py is a separate FastAPI app. Tool composition is poor, 4 arithmetic tools are duplicated across two servers (calculate_* and *_post variants). No security considerations visible (no input validation, no rate limiting). This is a study/demo project, not production-grade.
Add two numbers
Divide two numbers
Multiply two numbers
Subtract two numbers
Add two numbers together
Divide two numbers
获取所有列名
Multiply two numbers
Duplicate tools across servers: calculate_sum/subtract/multiply/divide (remote_server.py) and add_post/subtract_post/multiply_post/divide_post (api_server.py) perform identical arithmetic operations. LLMs will waste reasoning cycles choosing between them. Consolidate into one canonical set.
Generic descriptions repeated verbatim: 'Add two numbers together' appears on calculate_sum and add_post; 'Subtract two numbers' on calculate_subtract and subtract_post. Descriptions do not explain WHEN to use each tool, what makes them different, or what they return. Minimum 50 chars required; these are ~20 chars.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | 2026-07-28+ | v2 |
执行Pandas查询语句,查询幸福蓝海销售数据,注意查询表达式中不要包含df字符串,也不要包含分号,例如: "query": "销售方名称.str.contains('福州分公司')"
Subtract two numbers
Parameter descriptions missing: The 'query' parameter in the query tool has no description in the schema (only type: string). The description text in the tool docstring includes constraints ('no df, no semicolon') but these are not in the parameter schema where LLMs can parse them. Parameters a and b in arithmetic tools lack descriptions entirely.
No output schema documentation: Tools return TextContent with raw text (e.g., str(results) for query, response.text for remote calls). LLMs cannot infer what fields to expect or how to chain results to downstream tools. No documented return type or structure.
Weak error handling: Generic try/catch blocks return 'error: {str(e)}' with no guidance on recovery. The divide tools check for division by zero but return a plain error dict instead of a structured error with actionable next steps. No error classification (retryable vs fatal).