Loading ToolBench
Reference MCP server implementing the Service-Oriented Protocol Router (SOPR) pattern
The server implements 8 tools with explicit Zod schemas and descriptions. Strengths: comprehensive input validation with SafePathSchema preventing path traversal, reasonable bounds on text/array fields, use of enums for mode discrimination. Weaknesses: tool names lack action verbs (snap, check, learn, integrate, pulse, graph, cache, help are mostly nouns or vague verbs), descriptions are present but terse (10-60 chars), output schemas are completely undocumented, and error handling guidance is absent. The snap and learn tools have particularly vague names that do not signal their intent to an LLM before reading descriptions. Tool composition is reasonable (single responsibility mostly observed), but parameter documentation is inconsistent, many params like 'trigger', 'action', 'intent' lack concrete guidance on format/range.
Error & pattern cache operations. Modes: errors (error cache), patterns (pattern cache).
Code validation. Modes: quick, full, patterns, build, circular, security, coverage, orphans, health, evolution, integrations.
Dependency & file graph analysis. Modes: deps (dependency graph), files (file graph).
Tool discovery and documentation. Modes: tools (list all tools), status (current status), wire (wiring info), modes (mode details), thresholds (threshold config), decision (decision logic), all (everything).
External integrations. Modes: git (repository context), sentry (error tracking), github (PR/issue data).
Learning system. Modes: load (retrieve learnings), save (persist insight), search (query learnings).
Tool names lack action verbs. 'snap', 'learn', 'integrate', 'cache' are nouns or vague verbs that do not signal intent before description is read. LLMs cannot disambiguate from name alone. Should be: 'manage_snapshot', 'record_learning', 'sync_integrations', 'query_cache'.
Output schemas completely undocumented. The source shows input schemas via Zod (SnapInputSchema, CheckInputSchema, etc.) but no corresponding output type definitions or response examples. LLMs cannot plan multi-step operations or extract chaining IDs without knowing what fields each tool returns.
Descriptions are too terse (10-60 chars, under baseline 194 chars). Examples: 'Snapshot lifecycle...' (27 chars), 'Code validation...' (17 chars), 'Learning system...' (17 chars). LLMs need context on WHEN to use the tool and what each mode does. Expand to 100-150 chars explaining use case and mode behavior.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 47 | 2025-06-18+ | v2 |
| 2026-03-09 | F | 48 | - | v1 |
System health monitoring and service status reporting.
Snapshot lifecycle. Modes: start (begin task), check (validate), context (get context), end (complete), undo (revert + mask).
Parameter descriptions are generic or missing semantic detail. 'trigger', 'action', 'intent', 'context' lack concrete guidance on format, examples (without being literal), or constraints. E.g., 'trigger' (learn tool) says 'Trigger condition for the learning (used in save)' but does not explain what values are valid or what format is expected.
No error handling guidance or recovery paths. Tool descriptions do not explain what can go wrong, how to interpret failures, or what the LLM should do next. Example: 'check' mode 'full' might timeout or OOM on large codebases, no guidance on retry, scaling, or fallback.
No pagination or result limits documented. 'graph' tool can traverse dependency graphs with entryPoint and depth params, but no guidance on max results, pagination tokens, or when results might be truncated. Large graphs could blow context.
'pulse' tool is under-specified. Single mode 'health' with no parameters beyond mode. What does health return? Is it a status string, a structured report, an error? LLM cannot plan follow-up actions without knowing the response schema.
'cache' tool value parameter is typed 'unknown' with description 'JSON-serialized size capped at 1MB'. 'unknown' is not a proper schema type, should be 'any' (zod.any()) or a union of expected types (string | object | number). Unclear what LLM should pass.