Loading ToolBench
Model Context Protocol server for file operations and LLM diffs
Filesystem-mcp has complete input schemas for all 7 tools with proper JSON Schema structure (types, descriptions, required fields). However, tool descriptions are generic and lack LLM-optimization guidance. Parameter descriptions are minimal (10-30 chars) and do not explain constraints, formats, or when to use each tool. No output schemas are documented. The executeCommand tool is particularly risky, it accepts arbitrary commands with no whitelist enforcement visible in the code, despite the description claiming 'whitelisted'. Error handling returns basic text without recovery guidance. Tool naming is clear (verb_noun pattern), but composition issues exist: applyLLMChanges and diffWithLLM are LLM-specific wrappers that duplicate file operations already covered by readFile/createFile.
Apply LLM-generated content to a file
Create a new file with specified content
Compare an existing file with LLM-generated content
Execute a whitelisted command
Get the project facts file content
Read file content, optionally by line range
Update the project facts file with a new entry
executeCommand lacks input validation and whitelist enforcement. Code shows no command filtering despite description claiming 'whitelisted'. Accepts arbitrary commands, enabling command injection attacks. Parameter descriptions do not specify allowed commands or constraints.
No output schemas documented for any tool. LLMs cannot infer what fields to expect in responses, forcing them to parse unstructured text. Breaks tool chaining and wastes tokens on discovery.
Parameter descriptions are too brief (10-30 chars) and lack constraint information. E.g., 'format' in diffWithLLM lists options but does not explain when to use each. 'mode' in createFile shows octal value but not what permissions mean. LLMs cannot infer valid ranges or formats.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
Tool descriptions lack LLM-optimization guidance. Generic descriptions like 'Create a new file with specified content' do not explain when to use createFile vs applyLLMChanges, or what happens on path collision. No recovery hints for common errors.
Composition issue: diffWithLLM and applyLLMChanges duplicate and wrap readFile/createFile functionality. LLMs must reason about which variant to use, wasting tokens. Consider consolidating into single tools with optional LLM-specific parameters.