Loading ToolBench
Critical failures across all dimensions. Tool definitions are inferred from TypeScript UI code (toolDisplay.ts), not from actual MCP server registration. No visible input schemas with type definitions, only parameter names and descriptions are present. Descriptions are minimal (10-30 chars), far below the 50-200 char baseline for LLM-optimized tools. No output schemas documented. No error handling guidance. The codebase is a Tauri desktop app with React frontend, not an MCP server implementation. Tool definitions appear to be UI-layer mock definitions, not production MCP tool registrations.
Ask the user a question
Execute bash commands
Create a Vega visualization chart
Glob pattern matching on filesystem
List directory contents
Read file contents
Request filesystem access permission
Write file contents
Fetch web page contents
No input schemas visible in source code. Tool definitions appear to be UI mock objects, not MCP server tool registrations. Cannot verify JSON Schema with type definitions, constraints, or parameter validation.
Descriptions are 10-30 characters, far below the 50-200 char baseline. Examples: 'Read file contents' (18 chars), 'Write file contents' (19 chars). Too short to guide LLM tool selection or explain when to use each tool.
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields (e.g., does fs_list return file metadata, sizes, timestamps?).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 31 | <=2025-11-25 | v2 |
Search the web
Destructive tools (bash_exec, fs_write) lack error handling guidance, confirmation patterns, or dry-run support. No recovery instructions if a command fails or deletes unintended files.
bash_exec accepts arbitrary shell commands with no input validation, sanitization, or injection guards. LLMs can be tricked into passing malicious payloads (command injection, path traversal).