Loading ToolBench
Benchhand provides 12 tools with complete input schemas and descriptions. Naming follows verb_noun convention (system.health, workspace.open, filesystem.read). Descriptions are present and substantive (avg ~120 chars), exceeding the 10-char minimum. However, descriptions lack LLM-optimized guidance on WHEN to use each tool vs. alternatives, and error recovery paths are not documented. Output schemas are not visible in the provided source, only input schemas are evident. Parameter descriptions are adequate but lack constraint details (e.g., filesystem.search maxResults defaults to 100 but max is 1000; this is stated but not in parameter descriptions). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are present despite clear risk classifications (READ_ONLY vs WRITE). Composition is strong: tools chain well (workspace.open → filesystem.read), and IDs are consistent across calls.
Lists directory contents with pagination support
Applies multiple text edits to a file with SHA256 verification
Reads file content from a workspace with optional offset and byte limit
Searches files by glob pattern and optional text query with configurable depth and size limits
Writes content to a file with optional SHA256 verification and atomic durability
Resolves instruction documents for a workspace scope from configured providers
Retrieves an operation record by operation ID from the operation journal
Lists available skills in a workspace including shadowed and invalid entries
Output schemas not documented. Tool descriptions state what is returned (e.g., 'Returns system health status') but do not specify the response structure, field names, or types. LLMs cannot plan downstream calls or extract data without knowing response shape.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Risk classifications are provided in metadata (READ_ONLY vs WRITE) but not exposed in tool definitions. Agents cannot distinguish safe-to-retry operations from destructive ones without explicit hints.
Error handling and recovery guidance absent. No tool description explains what to do if a call fails (e.g., 'workspace not found, call workspace.list() to see available workspaces'). Agents have no recovery path on errors.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2025-06-18+ | v2 |
Reads the full content and metadata of a specific skill
Returns system health status including liveness, readiness, state, instance ID, RPC schema version, and storage integrity check result
Lists all workspaces registered in the system
Opens or creates a workspace at the specified path
Parameter constraint details missing from descriptions. filesystem.search accepts maxResults (default 100, max 1000) and maxBytes (default 256KB, max 1MB), but these bounds are not stated in parameter descriptions, only in tool description. LLMs cannot infer valid ranges from names alone.
Descriptions lack WHEN-to-use guidance. workspace.open and workspace.list both operate on workspaces but descriptions do not clarify when to call each. LLMs must infer intent from names alone, increasing selection errors.