Loading ToolBench
Single tool server with minimal but functional definition. Tool has explicit schema with typed parameter, description present but generic, lacks output schema documentation, no error handling guidance, no parameter constraints or format specifications. Parameter description is empty/missing. No complexity in composition. Median MCP server quality baseline suggests ~48 for this level of polish.
Generate a file tree for the specified directory, filtering files based on .gitignore. Traverses the filesystem and generates a JSON-formatted tree structure that preserves hierarchy.
Parameter 'directory' lacks description in tool registration. While the tool docstring mentions it, the actual parameter metadata (visible in @mcp.tool() decorator) shows no description field for the 'directory' parameter.
Output schema is not documented. Tool returns a JSON string representation of a tree structure, but LLMs cannot infer the schema of the returned JSON (field names, types, structure). This prevents downstream tool chaining and forces LLMs to parse unstructured output.
Error handling lacks recovery guidance. Tool returns {'error': 'directory not found'} as a JSON string, which is unstructured. Error should tell LLM what to do next (e.g., 'Directory not found. Verify the path exists and is readable.') and how to proceed.
Parameter 'directory' has no constraints documented. Should specify: must be an absolute or relative path; must exist; must be readable. Current description in schema is missing; LLM must guess valid input format.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 30 | - | v1 |
Tool description is generic and does not explain WHEN to use it instead of similar tools (e.g., filesystem traversal tools). Current description states what it does but not the use case or advantage (e.g., '.gitignore filtering' should be emphasized in the first line).