Loading ToolBench
MCP server that forces reasoning before any action via spec fingerprint tickets.
The server defines 4 tools with explicit schemas and descriptions visible in src/server.ts. However, quality is inconsistent across dimensions. Tool names follow verb_noun patterns (set_, log_, search_, revert_) which is good. Descriptions exist but are brief (50-100 chars) and lack context about when/why to use each tool. Schemas are present with typed properties, but parameter descriptions are terse and lack usage guidance. Most critically, output schemas are UNDOCUMENTED, the response format for each tool is not specified, forcing LLMs to infer structure. Error handling is basic (catch-all try-catch with generic messages). No tool annotations present. The tools operate on a file-system workspace model which introduces path traversal and injection risks that are not mitigated by input validation rules documented in descriptions.
Logs the outcome of a reasoning cycle to the memory log.
Reverts a reasoning transaction, removing the ticket and any associated learning logs. Use this to recover from a corrupted or unwanted state.
Searches the learning bank for past reflections within the current project.
Takes a task description and (optionally) a token budget, then returns a string instruction for an AI to reason adaptively.
Output schemas completely undocumented for all 4 tools. LLMs cannot plan downstream tool calls or extract data without knowing response structure.
Destructive tool 'revert_reasoning_transaction' has no confirmation, dry-run, or recovery guidance. Tool modifies state irreversibly (deletes ticket and logs) with no error recovery path or undo mechanism.
All tools accept 'workspace_path' (absolute filesystem path) as a parameter. No input validation or path traversal prevention documented. LLMs could be tricked into accessing arbitrary directories. Parameter descriptions lack format/constraint guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2025-06-18+ | v2 |
| 2026-03-09 | F | 42 | - | v1 |
Error handling is generic. All tools catch errors with 'Error in tool {name}: {message}'. No guidance to LLM on what to do next (retry? Ask user? Different tool?). Error messages are not actionable.
Tool descriptions are too brief (<100 chars) and lack context. 'Logs the outcome of a reasoning cycle' does not explain when to call this, what 'memory log' is, or what format 'learning' should take. Descriptions should be 100-200 chars with WHEN/WHY/WHAT guidance.
'search_learnings' tool returns JSON string but output format is not documented. Does it return an array? Objects with what fields? How is relevance scored (Fuse.js fuzzy match is in code but not mentioned in description)?
'set_reasoning_budget' tool returns 'a string instruction for an AI to reason adaptively' but does not specify format (JSON? Plain text? Special tokens like <think>?). Downstream AI cannot rely on structure.
Example: 'token_budget' is a number but no min/max specified. 'learning' is a string but no length limit or style guide. This invites invalid LLM input.