Loading ToolBench
Launch MCP servers with secrets from the OS keyring instead of plaintext mcp.json
mcp-secret-launcher presents a foundational STDIO MCP implementation with 6 tools focused on secret management and command execution. Tool definitions are visible in src/cli.rs with basic JSON schemas and descriptions present. However, the implementation exhibits significant quality gaps: (1) tool descriptions lack depth and actionability; (2) parameter descriptions are minimal to absent; (3) no error recovery guidance provided; (4) no output schemas documented; (5) no tool annotations (readOnlyHint/destructiveHint); (6) critical security concerns around secret handling not addressed in tool contracts. The 'Run' and 'AwsAuth' tools accept arbitrary command arrays, creating injection risk. The 'Set' tool mentions 'interactive prompt' but this is not a viable MCP pattern, tools must accept all required inputs as parameters. Average tool score: 42/100, below the 60+ baseline for production-ready tooling.
Authenticate via AWS SSO and launch the target command
Remove a secret from the keyring and manifest
Retrieve and display a masked secret
List all key names for a profile
Retrieve secrets and launch the target command
Store a secret in the keyring (interactive prompt)
Tool 'Set' declares '(interactive prompt)' in description but MCP tools must accept all inputs as parameters, interactive prompts are not viable in agent contexts. The secret value parameter is missing entirely from the schema.
'Run' and 'AwsAuth' tools accept arbitrary command arrays without documented validation, format constraints, or injection-protection guidance. This creates risk for command injection and makes the tool contract unclear to LLMs.
No output schemas documented for any of the 6 tools. LLMs cannot plan downstream calls or extract required fields (e.g., after Get, does the response contain {key, masked_value, full_value}? After List, is it [strings] or [{key, metadata}]?). Per pattern:tool and pattern:response-shaper, every tool must document return structure.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 38 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 44 | - | v1 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present despite clear risk classifications (WRITE, DESTRUCTIVE, READ_ONLY). Per current MCP spec (2026-07-28), tool annotations are the standard way to signal operation type to agents.
'AwsAuth' tool combines two responsibilities: authenticate via AWS SSO AND launch the target command. Per pattern:tool, each tool should do one thing. Should split into 'get_aws_credentials' (returns token/env) and separate 'run_command' invocation, or wire credentials as side effect.
Parameter descriptions are minimal or absent across all tools. For example, 'profile' parameter appears in 5 tools but is never described, is it a file path? A label? Must it exist first? Per pattern:tool-description, every parameter needs a 72-char baseline description explaining semantics and valid values.
No error recovery guidance provided. If a secret key is not found, does Get return empty string? Error object? Does it suggest alternatives per pattern:recovery-guide? Agents need actionable error messages to self-correct.
Tool descriptions are too short (30 - 70 chars). Rubric baseline is 194 chars (p10=34, p90=392). Current descriptions lack context on WHEN to use each tool vs alternatives, prerequisites, and expected outcomes.
Destructive tool 'Delete' has no dry-run, confirmation, or idempotency flag. Per pattern:confirmation-request, irreversible operations should support a confirm-before-execute pattern to prevent catastrophic errors by agents.
No pagination or limit guidance for List tool. Per pattern:paginated-result, tools returning lists should accept limit/offset and return total counts. If a profile has thousands of keys, result could balloon and exhaust context.