Loading ToolBench
Multi-backend environment variable and secrets manager with AES-256-GCM encryption, exposing vaulter tools, resources, and prompts via stdio transport
Vaulter MCP has 18 well-named tools with consistent verb_noun patterns (get_, list_, set_, delete_, plan_, apply_, create_, restore_, rotate_, check_). All tools have descriptions (avg ~80 chars, within baseline 34-392). Input schemas are present and typed for most tools. However, descriptions are terse and lack context on WHEN to use each tool, dependencies between tools (e.g., set_variable requires plan_changes + apply_changes), and recovery guidance for errors. Output schemas are not documented. Error handling is minimal, no guidance on retryability, user-fixable vs fatal errors, or recovery paths. Security-sensitive tools (rotate_encryption_key, restore_snapshot, apply_changes) lack confirmation/dry-run patterns despite being destructive.
Apply planned changes to backend (pushes local state to backend)
Check governance policies and guardrails for variables
Create a backup snapshot of an environment
Delete a snapshot
Delete a variable (local-first, requires plan/apply to push to backend)
Show diff between local and backend without creating plan artifact
Export variables from a project/environment/service
Retrieve audit log entries with optional filtering
Output schemas not documented. LLMs cannot infer what fields each tool returns, forcing them to guess downstream field names and risk broken tool chains.
Descriptions lack WHEN/WHY context and tool dependencies. E.g., set_variable says 'local-first, requires plan/apply' but doesn't explain the workflow or when to call plan_changes vs apply_changes. Agents cannot reason about multi-step sequences.
Destructive tools (restore_snapshot, delete_snapshot, rotate_encryption_key, apply_changes with force=true) lack confirmation/dry-run patterns. Agents can irreversibly delete snapshots or rotate keys without safeguards.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Retrieve current vaulter configuration and defaults
Get unified health and risk check for project/environment
Get a single variable value
List all environments defined in configuration
List all services in the monorepo
List all snapshots for an environment
Preview changes between local state and backend (generates plan artifact)
Restore a snapshot to backend
Rotate encryption key for a project/environment
Set a variable value (local-first, requires plan/apply to push to backend)
No error handling guidance. Tools do not document what errors are retryable, user-fixable, or fatal. LLMs cannot decide whether to retry, ask the user, or abort.
Descriptions for list_* tools are generic (e.g., 'List all environments'). Missing guidance on pagination, result limits, and when to call discovery tools first. Baseline: 194 chars avg; most here are 30-60 chars.