Loading ToolBench
A simple Model Context Protocol application for filesystem operations using local LLM (Ollama). Provides secure file read/write/edit operations with path validation and directory restrictions.
Solid foundation with clear tool names, complete schemas, and comprehensive descriptions. All 10 tools are explicitly registered with names, descriptions, and input schemas. Tool names follow verb_noun convention (read_file, write_file, etc.) and are action-oriented. Descriptions are detailed and explain WHAT, WHEN, and constraints. However, output schemas are not documented, LLMs cannot see what fields will be returned or plan downstream tool chains. Parameter descriptions are present but lack depth on constraints and formats. Error handling guidance is minimal. Security model relies entirely on path validation; no permission gates, scope declarations, or audit trail logging mentioned. Missing idempotency hints for retry-safe operations.
Create a new directory. Fails if the directory already exists or if the parent directory doesn't exist. Only works within allowed directories.
Get a recursive tree view of the directory structure. Provides an overview of all subdirectories and files in a hierarchical format. Only works within allowed directories.
Edit a file by specifying exact text to find and replace. The tool handles multiple edits in sequence. Supports a dry-run mode to preview changes before applying them. All edits are applied in order, and the tool provides a unified diff showing the changes made.
Get detailed metadata about a file, including size, creation time, modification time, access time, file type, and permissions. Only works within allowed directories.
List the contents of a directory, including file names, sizes, types, and modification times. Provides detailed information about each entry. Only works within allowed directories.
Output schemas are not documented. LLMs cannot see the structure of return values (e.g., what fields read_file, list_directory, or get_file_info return). This prevents planning of tool chains and forces LLMs to guess at field names in downstream operations.
Parameter constraints and formats are underspecified. For example, 'path' parameters lack guidance on absolute vs. relative paths, symlink handling, or supported encodings. 'pattern' in search_files lacks regex or glob format explanation. Without constraint details in descriptions, LLMs cannot predict valid input ranges.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 58 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Move or rename a file or directory. Works across directories as long as both source and destination are within allowed directories. Destination must not already exist.
Read the complete contents of a file from the file system. Handles various text encodings and provides detailed error messages if the file cannot be read. Use this tool when you need to examine the contents of a single file. Only works within allowed directories.
Read the complete contents of multiple files from the file system in a single request. Handles various text encodings and provides detailed error messages if any file cannot be read. Use this tool when you need to examine the contents of multiple files at once.
Search for files in a directory by name pattern. Recursively searches through subdirectories. Supports glob patterns for excluding certain files or directories from the search. Case-insensitive matching.
Write content to a file, creating it if it doesn't exist, or overwriting it if it does. Always uses UTF-8 encoding. Only works within allowed directories.
Destructive operations (write_file, edit_file, create_directory, move_file) lack confirmation or dry-run safety. edit_file supports dryRun, but write_file and move_file do not. Agents can accidentally overwrite or delete files without a preview-and-confirm flow. Missing destructiveHint annotations in tool definitions.
Error handling responses are not documented. Descriptions do not explain what errors can occur (file not found, permission denied, path traversal blocked, symlink attack prevented) or how to recover. LLMs receive raw error messages with no recovery guidance.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are visible in tool definitions. Modern MCP spec includes these hints to guide agent behavior. read_file should be marked readOnly; write_file, edit_file, create_directory, move_file should be marked destructive; read operations should be marked idempotent.
No scope declarations or permission gates. Tools do not declare what permissions they require (read:filesystem, write:filesystem, admin:filesystem). No runtime checks verify the caller has authority. An agent cannot be scoped to read-only.
No audit logging documented. Tool descriptions and source code do not mention logging of who called what tool with which parameters, when, and with what outcome. Filesystem operations are sensitive, lack of audit trails violates compliance best practices.
No batching support. Tools like search_files and read_multiple_files exist, but agents iterating over many files cannot batch operations efficiently. Missing array parameter variants for write, edit, or move to support multi-file operations in one call.