Loading ToolBench
HashiCorp Vault secret_provider plugin (dev.mcpg.secret.vault) for resolving secrets from Vault via vault:// URI scheme
Three tools with verb-based names (get, start_watch, cancel_watch) but critical gaps in schema visibility and parameter documentation. Tool descriptions are present but lengthy (200+ chars) and technical rather than LLM-optimized. Input schemas show parameter names and types but lack formal JSON Schema structure in visible code. No output schemas documented. The 'get' tool description is 380+ chars, exceeds best-practice range (10-1024 chars, optimal 50-200). Parameter descriptions exist but are sparse. No error handling guidance visible. Schemas appear inferred from Rust type signatures rather than explicitly registered.
Reclaim the leaked WatchState pointer and drop it, which fires WatchState::drop and aborts the background watch task. Safe iff handle was produced by start_watch and hasn't been dropped already.
Resolve a vault://<path>#<field> reference. The request maps to GET /v1/{path} (the URI's path is the full backend path including the mount segment); the response's data envelope is unwrapped and the configured field anchor (or default_field when the URI omits one) is extracted. KV v2 returns { "data": { "data": {...}, "metadata": {...} } } — a double data wrap. We detect KV v2 by the presence of a nested data object and unwrap it; non-KV-v2 paths (dynamic creds, PKI) skip the nested unwrap.
Start a poll-mode or native rotation watch on secret_ref. The plugin spawns a tokio task that re-reads the secret on a configured interval (poll strategy) or subscribes to sys/events/subscribe over WebSocket (native strategy, Vault 1.13+); on a KV v2 metadata.version bump it emits a SecretRotationWire JSON through emit_event. Returns a WatchHandleBox pointer for later cancellation.
No visible JSON Schema definitions for tool inputs. Parameter types inferred from Rust signatures; formal schema registration not shown in source.
Tool descriptions exceed 200 chars and use technical jargon ('KV v2', 'double data wrap', 'WatchHandleBox pointer'). LLM-optimized descriptions should be 50-200 chars and avoid implementation details.
No output schemas documented. Callers cannot infer what fields to expect from tool responses, breaking downstream tool chaining.
Parameter 'emit_event' in start_watch typed as 'function', not a standard JSON Schema type. Unclear how LLM should invoke or what signature is expected.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 38 | <=2025-11-25 | v2 |
No error handling guidance. Descriptions do not explain failure modes, recovery steps, or when to retry. E.g., what happens if Vault is unreachable or secret not found?