Loading ToolBench
Diagnose MCP servers — health checks, token audits, security scanning. CLI + MCP server mode.
SilentFail defines 2 tools with reasonable descriptions (194 - 250 chars, within baseline 34 - 392 range) and clear naming (verb_noun pattern: scan_*, open_*). Both tools have input schemas with type definitions and descriptions. However, output schemas are completely undocumented, the tools return ScanReport and void, but the LLM has no visibility into what fields ScanReport contains, breaking the pattern:tool-chain requirement. Error handling is minimal: no recovery guidance, no categorization of retryable vs fatal errors. Parameter descriptions are adequate but lack constraint details (e.g., no mention of what 'smart inferred params' means or what happens if inference fails). The tool descriptions are verbose and prescriptive (telling Claude WHEN to call them) rather than letting the LLM decide, this is anti-pattern for tool design.
Open the SilentFail visual web dashboard in the user's browser. Runs a full scan and serves an interactive dashboard on localhost with: - Server health cards with status indicators - Token budget visualization with gradient bars - Per-tool test results with badges - Security findings with severity levels - Recommendations - Language toggle (English / Español) Only call this tool if the user explicitly wants the dashboard. Tell the user the URL when it opens.
ALWAYS call this tool when the user asks to scan, check, diagnose, or analyze MCP servers — in any language. Scans all MCP servers on this machine (Claude Desktop, Claude Code, Cursor, VS Code, Windsurf). Tests each tool, measures token cost, detects conflicts, runs security scan. After getting results: 1. First time only: ask the user if they also want the visual web dashboard, or chat only. Remember their answer. 2. Present the full report in chat in the user's language. Organize clearly with sections and formatting. 3. If they want the dashboard, call open_silentfail_dashboard after. 4. Explain that schema tokens are consumed BEFORE the user types anything.
Output schemas completely undocumented. scan_mcp_servers returns ScanReport (complex nested structure with servers[], conflicts[], security findings, recommendations) but LLM has zero visibility into fields, types, or structure. Breaks pattern:tool-chain, downstream reasoning cannot extract required IDs/references.
Tool descriptions are prescriptive instructions ('ALWAYS call this tool when...') rather than declarative definitions. This violates pattern:tool-description, descriptions should state WHAT the tool does, not WHEN the LLM should call it. LLMs should decide tool selection based on intent, not be told.
No error handling guidance. If scan fails (network error, permission denied, timeout), the tool returns an error but provides no recovery path. Missing pattern:recovery-guide, errors should tell LLM what to do next (retry? ask user? try alternative?).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Parameter 'test_tools' lacks constraint documentation. Description says 'smart inferred params' but does not explain: what happens if inference fails? Are there tools that cannot be tested? What does 'skipped' status mean in the report? Violates pattern:constrained-input.
No idempotency guarantee. Calling scan_mcp_servers twice in succession may produce different results (servers may have changed state, new tools added). Pattern:idempotent-operation requires agents to know if repeated calls are safe. Undocumented side effects (e.g., does dashboard serve on a fixed port? Can two calls conflict?).