Loading ToolBench
A gateway for managing Model Context Protocol (MCP) servers, including tool deployments, agent management, and execution of built-in tools (bash, read_file, write_file).
The MCP Gateway implements three built-in tools (bash, read_file, write_file) with clear, well-documented schemas and good parameter descriptions. Tool names follow verb_noun convention (builtin_bash, builtin_read_file, builtin_write_file) and are action-oriented. All three tools have documented input schemas with type definitions and parameter descriptions. However, there are gaps in output schema documentation, no explicit error handling guidance in tool descriptions, and missing security/risk annotations in the schema itself (though risk classification is present in metadata). The authorization layer (BuiltinToolAuthorizer) demonstrates proper permission gating, which is a security strength. Parameter descriptions clearly explain path validation and constraints. Overall, the tools are solidly defined but lack some completeness around output structure and error recovery guidance that would elevate them to A-grade.
Run a shell command in the session's working directory. Output is captured (stdout, stderr, exit code).
Read a UTF-8 text file relative to the session's working directory.
Write (or overwrite) a UTF-8 text file relative to the session's working directory.
Output schemas not documented in tool definitions. Tool descriptions state what is returned (stdout, stderr, exit code for bash; file content for read; success indicator for write) but formal output schema structure is not present in the registration. LLMs cannot reliably parse or chain outputs without documented schemas.
Error handling guidance missing from tool descriptions. While the code implements proper authorization checks (BuiltinToolAuthorizer), tool descriptions do not explain failure modes (e.g., 'unauthorized', 'path outside session directory', 'command timeout') or recovery paths. LLMs cannot determine if an error is retryable or what to do next.
No tool annotations (destructiveHint, readOnlyHint, idempotentHint) in schema. The metadata includes risk classification (IRREVERSIBLE, READ_ONLY, WRITE) but these are not exposed in the MCP tool schema as tool annotations per the current spec. LLMs cannot self-classify tool safety without explicit annotations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 68 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
bash tool accepts unbounded timeout_seconds (1-120 stated in description, but no explicit max validation shown in schema). While the description constrains it, the schema should enforce min/max bounds via JSON Schema minimum/maximum keywords.
No idempotency guarantees documented. bash tool is inherently non-idempotent (running 'rm -rf /data' twice has different outcomes). Tool description should warn agents of this and recommend confirmation patterns for destructive commands.