Loading ToolBench
A deterministic MCP harness protocol for governed tool execution with pre-model routing, policy validation, and repeatable scenarios. Provides both MCP stdio server and local HTTP API with Studio UI.
Three tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. Tool names are action-oriented (invoke, inspect, scenario) but descriptions lack depth on when/why to use each tool. Parameters have types but most lack detailed descriptions explaining constraints, formats, or dependencies. No error handling guidance. Output schemas are not documented, LLMs cannot predict response structure. invoke and scenario accept JSON-encoded parameters as strings, forcing LLMs to construct JSON manually, error-prone and violates chat-data-model pattern. No pagination, no result limits documented. Security model unclear (what does 'governed' mean? what policies apply?).
Inspect the current state for one target or the whole harness.
Validate and execute a governed tool call.
Run a predefined deterministic scenario.
Parameters lack detailed descriptions. 'target' and 'action' in invoke are vague, no explanation of valid values, format, or how they map to the catalog. LLMs cannot infer constraints.
invoke accepts 'parameters_json' as a string, forcing LLMs to construct JSON manually. Violates chat-data-model pattern, users say 'call X with foo=bar', not 'call X with {"foo":"bar"}'. Error-prone and wastes tokens on JSON serialization.
Output schemas not documented. invoke, inspect, and scenario return JSON strings, but LLMs cannot predict field names, types, or structure. Breaks downstream tool chaining and forces parsing guesswork.
No error handling guidance. If invoke fails (policy violation, invalid target, malformed JSON), the response does not tell the LLM what to do next or how to self-correct. Blocks agent recovery.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
Security model opaque. 'Governed tool call' and 'policy' are mentioned but not explained. What policies apply? What happens if a call violates policy? How does the agent know what is permitted? No scope declarations or permission gates visible.