Loading ToolBench
A Flask-based MCP server that provides file management tools (list, read, write, delete, search) with comprehensive JSON-RPC logging
This server has fundamental quality issues across naming, descriptions, parameter documentation, and schema completeness. All 5 tools are present with basic schemas, but descriptions are minimal (10-15 chars on average, well below the 194-char baseline), parameter types are defined but lack descriptions entirely, and output schemas are not documented. The tutorial nature does not excuse production-quality gaps. Tools like 'list_files' and 'read_file' follow naming conventions, but parameter documentation is sparse. Error handling is not evident from the definitions. This is a typical D-grade community tutorial server.
Delete a file by name
List all files in the directory
Read the contents of a file
Search for a keyword in a file and count occurrences
Write text to a file (creates or overwrites)
Descriptions are critically short (10-15 chars) vs. the 194-char baseline. 'List all files in the directory' and 'Read the contents of a file' lack context about when to use them, what they return, and dependencies.
No parameter descriptions visible. All tools with required parameters (read_file, write_file, delete_file, search_file) lack descriptions for 'filename', 'text', 'keyword'. The description 'filename' is not enough, it should explain format constraints, whether absolute/relative paths are allowed, and what happens if the file does not exist.
No output schemas documented for any tool. An array of filenames? Objects with name, size, type? read_file returns a string? Object with {filename, content, encoding}? Without documented return types, agents cannot plan downstream operations or extract the right fields.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 44 | 2025-06-18+ | v2 |
| 2026-03-09 | F | 31 | 2025-06-18+ | v1 |
Destructive operations (write_file, delete_file) lack error handling and confirmation patterns. An agent calling delete_file with the wrong name deletes the file immediately with no recovery option. write_file overwrites silently with no undo. No error messages documented for conflict scenarios (file not found, permission denied, disk full).
Missing tool annotations. Per current MCP spec (2026-07-28), tools should declare readOnlyHint, destructiveHint, and idempotentHint. list_files, read_file, and search_file are read-only; delete_file is destructive; write_file is not idempotent (repeated calls with same text are idempotent, but parameter changes risk overwrites). Annotations are visible in the evaluation context (Risk field marks some tools correctly), but not declared in the server code.
Parameter 'filename' in multiple tools does not specify format constraints. Is it a relative path ('file.txt', 'dir/file.txt')? Absolute ('/tmp/file.txt')? URI ('file:///tmp/file.txt')? Can it contain '..' for directory traversal?
No pagination or result limits documented for list_files. If the directory contains thousands of files, does the tool return all of them? Agents cannot reason about massive responses.
search_file 'keyword' parameter lacks description. Is it a regex pattern? Substring match? Case-sensitive? Without constraint details, agents may pass regex when the tool expects literal strings, or vice versa.