Loading ToolBench
Roo MCP Server provides 4 filesystem tools with basic schemas and minimal descriptions. Tool names follow verb_noun convention (fs_read, fs_write, fs_list, fs_search), which is appropriate. Input schemas are present and use Zod with proper types, but lack critical metadata like bounds, enums, and format constraints. Output schemas are entirely undocumented, callers don't know what structure to expect from fs_list or fs_search. Error handling in the service layer catches and wraps exceptions generically, but tools don't expose recovery guidance to the LLM. No security considerations are apparent (path traversal risk in fs_read/fs_write is not mitigated with sandboxing), and no parameters are marked as sensitive. Tool composition is sound, each tool has one clear responsibility, but descriptions don't explain WHEN to use each tool or what downstream actions they enable.
List files in directory
Read file contents
Search files by content
Write to file
Tool descriptions are critically underdeveloped. All 4 tools have descriptions under 30 characters ('Read file contents', 'Write to file', 'List files in directory', 'Search files by content'), failing to explain WHAT the tool does beyond a bare action, WHEN to use it instead of related tools, or what structure it returns. Baseline is 194 chars; these descriptions waste the opportunity to guide LLM tool selection.
Output schemas are completely undocumented. fs_list and fs_search return JSON-stringified arrays (seen in execute: JSON.stringify(...)) but callers cannot know the structure without reading the code. The rubric baseline is 100% of A+ tools have documented return types. LLMs will struggle to compose downstream operations.
Path traversal vulnerability: fs_read and fs_write use path.resolve(process.cwd(), filePath) which allows relative paths like '../../../etc/passwd' to escape the intended directory. No sandboxing or allowlist is enforced. Agents could be tricked into reading/writing sensitive files outside the project.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 42 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 8 | - | v1 |
fs_write lacks any confirmation or dry-run mechanism. Agents can irreversibly overwrite or delete files without a safety check. Rubric pattern:confirmation-request requires irreversible operations to support dry-run or confirmation.
Parameter descriptions are generic and lack actionable constraints. 'File path to read' does not explain format, length limits, or whether absolute/relative paths are allowed. 'Search pattern (regex)' does not warn that invalid regex will throw or provide examples. Baseline: param descriptions should be 72 chars average with explicit constraints.
fs_list 'recursive' parameter and fs_search 'filePattern' parameter lack enum constraints. 'recursive' could explicitly state 'boolean, default false'. 'filePattern' describes the format as 'e.g. *.ts' (example in description), violating the rule: 'Replace examples with enum/format constraints; LLMs latch onto examples.'
Error handling uses generic catch-all: 'if (error instanceof Error) { throw new Error(...) } else { throw ... Unknown error }'. The wrapped errors do not categorize as retryable/user-fixable/fatal, nor do they include the invalid value or actionable next steps. Rubric baseline: errors must tell LLM what to do next.
fs_search regex pattern and filePattern parameters are not validated before use. Invalid regex (e.g. '[unclosed') will throw at runtime without helpful guidance. Rubric: 'Do not assume LLMs will follow constraints. Validate inputs early and return clear error messages.' E.g., 'Invalid regex: got "[unclosed", check for unmatched brackets.'
fs_list and fs_search have no pagination or result-limit enforcement. A recursive search of a large codebase could return thousands of files, exhausting context windows. Rubric baseline: 'cap results at 20-50 and offer pagination.' No limit is stated in descriptions or parameters.