An MCP server + CLI for AI-agent-safe secret management with encryption, masking, and self-healing rotation
Keysmith has well-structured tool definitions with clear security-focused descriptions and proper JSON schemas. All 5 tools have descriptions (avg 150 chars, within baseline 34-392 range) and input parameters with type annotations. However, output schemas are not formally documented, parameter descriptions lack constraint details (ranges, formats, enums), and error handling lacks recovery guidance. The security model is thoughtfully designed (masked values, no plaintext in transcripts) but not reflected in tool annotations (readOnlyHint, destructiveHint). Naming is verb-based and clear (list, get, put, rotate, delete), following conventions well.
Delete a secret key
Read a single secret's MASKED value. MCP never returns plaintext — to use a value without reading it, redeem it with `keysmith run` on the host.
List all secret keys with MASKED values (plaintext never shown)
Store a secret. Pass value_file (a path to a file containing the value) to avoid putting plaintext in the transcript.
Generate a new random secret for key and store it. Returns the masked value only; plaintext is never returned over MCP.
Output schemas not documented. Tool results return TextContent with unstructured strings (e.g., 'stored key (masked: ****)'), but LLMs cannot parse structured fields for downstream chaining. No documented return type schema.
Parameter descriptions lack constraint details. 'length' in rotate has no min/max bounds (default 32 stated, but no range). 'key' parameters have no format/pattern guidance. LLMs may pass invalid values without explicit constraints.
Error handling lacks recovery guidance. Errors like 'key not found' or 'set failed' do not suggest next steps (e.g., 'Try list() to see available keys'). Agents cannot self-correct without explicit guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Tool annotations missing. delete is destructive and put is a write operation, but no destructiveHint or idempotentHint annotations present in tool definitions. Agents cannot infer operation safety.
delete tool lacks confirmation/dry-run pattern. Irreversible operations should support a confirmation step to prevent accidental data loss. No dry_run parameter or confirmation mechanism.