Loading ToolBench
A MCP server to retrieve data from source code of real code projects. It provides tools for retrieving actual and relevant context when right data is specified.
Scaffold MCP has 3 tools with visible definitions in src/mcp/server.py. All tools are registered via @mcp.tool decorator with FastMCP. However, the server has significant gaps in definition quality: descriptions are present but lack actionable detail for LLM tool selection; input schemas are minimal or absent for most tools; output schemas are completely undocumented; no error handling guidance; no parameter type validation hints. The tool descriptions read more like internal documentation than LLM-optimized prompts (pattern:tool-description baseline: 10-1024 chars, ideally 50-200 for clarity). Tool naming is adequate but generic (all start with 'get_', which is good), but descriptions do not explain WHEN to use each tool vs. alternatives or what the LLM should do with the response.
Return all classes names and its path in project. Better call this tool than use grep for search exact names
Return all functions names and its path in project. Better call this tool than use grep for search exact names
Find code entity information in project using relevant name and return information (code, relationships, vectorchunks). You need just specify correctly what entity you are interesting for using entity_name field
Output schemas completely undocumented. Tool definitions show return type as dict with generic 'response' key, but LLM has no visibility into the actual structure (list of objects? flat array? nested fields?). Per pattern:tool and pattern:response-shaper, documented output schemas are critical for downstream chaining and context-efficient reasoning.
Descriptions lack actionable LLM guidance. No explanation of WHEN to use each tool, what the agent should do with results, or how tools chain together. Per pattern:tool-description baseline (avg 194 chars for A+ tools), descriptions should state WHAT it does, WHEN to use it, and WHAT to expect back. Current descriptions are too terse and procedural.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 56 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Input parameter 'entity_name' lacks validation constraints. No description of acceptable formats (full names? partial matches? regex? case-sensitive?). Per pattern:constrained-input, LLM needs explicit constraints to avoid passing invalid values.
No error handling or recovery guidance. If entity not found, API returns what? Empty response? Error dict? What should LLM do next? Per pattern:recovery-guide, errors must tell the agent what to do: 'Not found. Try get_all_functions_nodes_names() to list available entities.'
Two discovery tools (get_all_functions_nodes_names, get_all_classes_nodes_names) suggest potential overlap or poor decomposition. Per pattern:tool, each tool should do exactly one thing. Consider: (a) merging into a single get_all_code_entities tool with an 'entity_type' enum param, or (b) clearly documenting the distinction and when to use each.
No pagination or result limits documented. If project has thousands of functions/classes, unbounded results will blow LLM context window. Per pattern:paginated-result and mxe:enforce-result-limits, tools returning lists should accept limit/offset and document maximum results returned.