Loading ToolBench
MCP server for Sieve — local AI chat history secret scanner for macOS
Sieve MCP has well-defined tool names (verb_noun pattern: sieve_health_status, sieve_findings_list, etc.) and clear, security-conscious descriptions that explicitly state what data is NOT returned (e.g., 'never echoes input', 'raw values are never returned'). All 9 tools have input schemas with proper JSON Schema structure. However, several parameters lack descriptions (e.g., sieve_vault_run's 'fingerprints' array has a description, but many optional params in other tools are undescribed). Output schemas are not documented, LLMs cannot predict response structure. Error handling is absent from visible code; the stub returns only a generic message. The security posture is strong (no secrets in params, read-only operations), but composition could be tighter, some tools (sieve_findings_list, sieve_scan_history) accept optional 'limit' params but lack min/max constraints or pagination guidance.
Checks whether text contains secrets. Returns boolean only — never echoes input or detected values.
Lists findings from the most recent scan. Previews are redacted — no plaintext secret values.
Returns Sieve service health and finding counts.
Lists scanner health issues: unreadable sources, permission errors, parse failures.
Redacts secrets from text, replacing values with sieve:// placeholders. Raw values are never returned.
Returns past scan job summaries.
Returns status of the current or most recent scan: idle, running, completed, or failed.
Output schemas not documented. LLMs cannot predict response structure (fields, types, pagination). Tools like sieve_findings_list and sieve_scan_history return lists but lack documented pagination, total count, or field definitions.
Numeric parameters lack min/max constraints. 'limit' params in sieve_findings_list (default 50) and sieve_scan_history (default 10) have no documented bounds, allowing LLMs to pass absurd values (e.g., limit=999999).
Error handling absent from visible code. The introspection stub returns a generic message for tools/call; no guidance on retryability, user-fixable errors, or recovery steps. Production binary behavior unknown.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Verifies scanner rules and transcript source coverage are healthy.
Executes a command with macOS Keychain-resolved secrets injected as env vars. Returns exit code and line counts only — stdout/stderr content is never returned.
sieve_vault_run is a WRITE tool (executes shell commands) but lacks confirmation/dry-run pattern. Agents could accidentally run destructive commands without safeguards.
Protocol version hardcoded as '2024-11-05' in initialize response. Current spec is 2026-07-28. This indicates the server has not been updated to track protocol evolution.