Loading ToolBench
An MCP server that provides access to a vault service for secure secret management and retrieval
Canterbury exposes 4 vault tools with basic naming (verb_noun pattern) and minimal descriptions. All tools have input schemas with typed parameters and descriptions. However, descriptions are extremely terse (10-20 chars), falling well below the 50-200 char baseline for LLM-optimized tools. No output schemas are documented. Error handling, recovery guidance, and parameter constraints are absent. The vault_put tool accepts an untyped 'secret' object parameter, creating ambiguity about expected structure. No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. Composition is reasonable (each tool does one thing), but descriptions lack context about when to use each tool vs alternatives.
Delete a secret from the vault
Retrieve a secret from the vault by path
List secrets at a given vault path
Store a secret in the vault at a given path
Descriptions are critically short (10-20 chars). LLMs cannot determine when to select each tool or what distinguishes vault_get from vault_list. Baseline is 50-200 chars.
vault_put 'secret' parameter is untyped object with no schema constraints. LLMs cannot infer what fields are required or what structure to pass. Should define a schema with required fields and types.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., does vault_list return paths, metadata, or full secrets?).
No tool annotations despite clear risk levels. vault_delete is DESTRUCTIVE and vault_put is WRITE, but no destructiveHint or readOnlyHint annotations present in schema.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
No error handling guidance. If a path does not exist or permission is denied, LLMs receive no recovery hints. Should include actionable error messages and retry guidance.