Loading ToolBench
An MCP server implementation for Code Assistant, providing tools for code analysis, file exploration, command execution, and Git operations
Scoring was not performed
No input schemas visible in source code for any tool. Per hard scoring rule: 'If a tool has NO input schema at all: its schema score MUST be 0.' All 15 tools lack visible JSON Schema definitions with type constraints and property descriptions.
Tool descriptions are minimal (10-15 characters: 'Read files from the project', 'List files in a directory'). No description answers the critical questions: What does it do? When use this vs similar tool? What does it return?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 39 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 19 | - | v1 |
No evidence of parameter descriptions. Rubric requires every input parameter to have a non-empty description. Without parameter-level docs, LLMs cannot determine valid values, formats, or constraints. This forces LLMs to guess and causes misuse.
Destructive tools (delete_files, execute_command, spawn_agent, edit, write_file) lack error handling guidance or confirmation patterns. Per pattern:confirmation-request, irreversible operations should support dry-run or explicit confirmation. No evidence of recovery guides or categorization (retryable vs fatal vs user-fixable).
No output schema documentation visible. Rubric requires: 'Document the output schema. LLMs need to know what fields to expect.' Without output schemas, agents cannot plan downstream tool calls or extract required data (e.g., what fields does list_files return? How many results?)
Vague tool names on state-modification tools. 'edit' is ambiguous, does it apply to files? Code? Configuration? LLM cannot distinguish from 'write_file' without clear descriptions. Rubric: 'The tool name alone should convey what happens when called.' Current names force guessing.
No pagination or result-limiting patterns documented. Tools like list_files, search_files, list_projects likely return unbounded results.
web_fetch and web_search offer no guidance on parameter constraints, rate limits, timeout behavior, or structured response format. Rubric: 'Set explicit timeouts for tools that call external services.' No evidence of timeout handling or partial-failure strategies.
execute_command presents severe security and safety risks with no documented constraints. No evidence of allowlist, command validation, sandboxing, or user confirmation. Rubric: 'Treat all agent-provided input as untrusted. Sanitize against SQL injection, command injection, and path traversal.'