Loading ToolBench
MCPunk defines 2 tools with basic schemas and descriptions, but has significant gaps in description quality, parameter documentation, and composition. The 'get_a_joke' tool is trivial (single string param, vague description). The 'configure_project' tool has a long, complex description but lacks actionable parameter guidance and schema documentation for its response structure. Parameter descriptions are minimal ('animal', 'Root path of the project', generic). No error handling guidance, no pagination for list operations, and no evidence of output schema documentation. The server demonstrates awareness of MCP but lacks the rigor expected of production tools. Average per-tool score: 48 (get_a_joke: 35, configure_project: 61).
Configure a new project containing files. Each file in the project is split into 'chunks' - logical sections like functions, classes, markdown sections, and import blocks. After configuring, a common workflow is: 1. list_all_files_in_project to get an overview of the project (with an initial limit on the depth of the search) 2. Find files by function/class definition: find_files_by_chunk_content(... ["def my_funk"]) 3. Find files by function/class usage: find_files_by_chunk_content(... ["my_funk"]) 4. Determine which chunks in the found files are relevant: find_matching_chunks_in_file(...) 5. Get details about the chunks: chunk_details(...) Use ~ (tilde) literally if the user specifies it in paths.
Get a really funny joke! For testing :)
Descriptions lack sufficient detail to disambiguate tool purpose and when to use them. 'Get a really funny joke! For testing :)' is too informal and doesn't explain prerequisites, return structure, or use case clarity. 'configure_project' description is verbose (300+ chars) but buries actionable details and does not clearly state what happens after configuration.
Parameter descriptions are generic and lack constraints. 'animal' param has no guidance on valid inputs, expected format, or what happens if invalid data is passed. 'Root path of the project' does not specify absolute vs. relative path, tilde expansion behavior (despite the tool description mentioning it), or what constitutes a valid project root.
Output schemas are not documented. LLMs cannot determine what fields to expect from tool responses, making it impossible to chain tools or extract the right data. 'configure_project' specifically, what does it return? A project ID? A status? File tree structure? This forces agents to guess.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
No error handling or recovery guidance. If 'get_a_joke' fails or 'configure_project' cannot access a path, what should the LLM do next? No hints, no actionable error messages defined.
Tool composition is broken. 'configure_project' description mentions downstream tools (list_all_files_in_project, find_files_by_chunk_content, etc.) that are NOT REGISTERED as MCP tools. Either these should be registered as tools, or the description should not reference them. LLMs will try to call tools that don't exist.
Parameter naming is inconsistent with patterns. 'animal' lacks a noun suffix (should be 'animal_type' or match an enum). 'project_name' is clear but 'root_path' could benefit from validation constraints in the description (absolute vs. relative, tilde handling).
Input schema for 'get_a_joke' has a maxLength constraint (20 chars) but this is not explained in the parameter description. The LLM will not know why the constraint exists or how to adapt if input exceeds it.