Loading ToolBench
Local-first permission policies, approvals, and redacted audit for MCP tool calls
ToolPermit defines 6 tools across demo, test, and fixture servers. Tool names are action-oriented (write_demo, read_demo, echo, add), but descriptions are minimal (10-34 chars, well below the 50-200 char LLM-optimized baseline). Input schemas are present with type declarations and parameter descriptions, but lack critical constraints (enums, ranges, format specs). No output schemas documented. Error handling is absent, tools do not guide recovery or categorize failures. The write_demo tool accepts a token parameter with a hardcoded default, which is a security anti-pattern. Overall, definitions are skeletal: functional but not production-ready.
Add two integers.
Add two integers.
Return text unchanged.
Return text unchanged.
Read UTF-8 text below the explicitly configured disposable demo directory.
Write UTF-8 text below the explicitly configured disposable demo directory.
Descriptions are critically short (10-35 chars). LLM-optimized descriptions should be 50-200 chars, explaining WHAT the tool does, WHEN to use it, and any prerequisites. Current descriptions lack context for tool selection.
No output schemas documented. LLMs need to know what fields to expect in responses to plan downstream calls and extract data correctly. write_demo and read_demo return strings; echo and add return primitives, all lack formal output type declarations.
write_demo accepts a 'token' parameter with hardcoded default 'demo-not-a-secret'. This is a security anti-pattern: credentials should never be tool parameters. Use server-side secret injection via environment variables instead.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2026-07-28+ | v2 |
No error handling or recovery guidance. Tools do not validate inputs (e.g., path traversal in write_demo/read_demo is checked in code but not documented), do not categorize errors as retryable/user-fixable/fatal, and do not guide LLMs on next steps after failure.
Duplicate tool names (echo and add appear twice across fixture servers). This creates ambiguity for LLMs and violates the composition principle of one canonical tool per action. Consolidate or rename variants.