Loading ToolBench
Azure Data Lake Storage Gen2 Model Context Protocol (MCP) service
Server has 15 tools with consistent schemas and descriptions. Tool names follow verb-noun conventions (list_, create_, delete_, upload_, download_, rename_, get_, set_, directory_*). All tools have descriptions (avg ~70 chars) and input schemas with typed parameters. However, descriptions are terse and lack context on WHEN to use tools or prerequisites. Parameter descriptions are minimal (avg ~45 chars). Output schemas are not documented, tools return dataclass-serialized dicts but the LLM cannot predict structure without reading source. Error handling returns structured responses but lacks recovery guidance. Security checks for read-only mode exist but are not reflected in tool annotations. Overall: competent but missing optimization for LLM reasoning and composition.
Create a new directory in the specified filesystem
Create a new ADLS2 filesystem (container)
Delete a directory from the specified filesystem
Delete an ADLS2 filesystem
Check if a directory exists in the specified filesystem
Get all paths under the specified directory
Download a file from ADLS2
Tool descriptions are terse (avg ~70 chars) and lack context on WHEN to use a tool vs alternatives. Example: 'List all filesystems in the storage account' doesn't explain prerequisites (e.g., 'Requires Azure authentication') or when to call this before other operations.
Output schemas are not documented. Tools return dataclass instances serialized to dicts (e.g., DirectoryResponse with fields path, success, error), but the LLM has no schema reference to predict structure. Without documented outputs, downstream tools cannot safely compose.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | F | 10 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 55 | - | v1 |
Check if a file exists in the specified filesystem
Get metadata of a file in the specified filesystem
Get properties of a file in the specified filesystem
List all filesystems in the storage account
Rename/move a directory within the specified filesystem
Rename/move a file within the specified filesystem
Set a single metadata key-value pair for a file
Upload a file to ADLS2
Parameter descriptions are minimal and lack constraints. Example: 'upload_file' parameter 'upload_file' has description 'Path to the file to upload (relative to UPLOAD_ROOT)' but does not specify: max file size, allowed extensions, or behavior if file exists. LLMs cannot infer these constraints.
Error handling returns structured responses but lacks recovery guidance. Example: DirectoryResponse includes 'error' field as free-text string. A failed operation returns 'Failed to create directory' with no hint on WHY or what to do next. Pattern requires: 'User not found. Try search_users() with a partial name.'
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) declared. Destructive tools (delete_filesystem, delete_directory) and read-only tools (list_filesystems, file_exists) should be marked with annotations so agents can reason about safety. Currently, the LLM must infer from description text alone.
No pagination support visible. 'directory_get_paths' accepts a recursive flag but does not document result limits or pagination. If a directory contains thousands of items, the response could exhaust context. Should include limit, offset, and total_count fields.
No dry-run or confirmation mechanism for destructive operations. delete_filesystem and delete_directory are irreversible. Pattern requires: support a dry-run step or explicit confirmation to prevent accidental data loss when agents make mistakes.
No natural identifiers or fuzzy matching. Tools require exact filesystem/directory names. If a user says 'upload to my data lake', the agent must know the exact filesystem name. No search or suggestion mechanism for discovering available filesystems by partial name match.