Loading ToolBench
MCP server that exposes workspace tasks through STDIO transport, enabling AI agents to discover and execute tasks from projects
Server has 15 tools with consistent verb-noun naming (list_, run_, start_, get_, search_, stop_). Descriptions are present and moderately detailed (avg ~120 chars), explaining WHAT and WHEN. However, input schemas are largely absent from visible code, only 2 tools (define_shell_block, run_shell_command) show explicit parameter schemas. Most tools lack documented output schemas. Error handling guidance is minimal. Parameter descriptions exist but lack constraints (enums, ranges, formats). No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. Composition is sound, tools chain well (run_task → wait_run → get_run_status), but schema documentation is the critical gap.
Define a reusable shell command block with exact argv; no shell interpretation.
Get persisted metadata for one task run.
Read a stdout or stderr log page.
Get a run snapshot.
List task projects.
List recent runs.
List tasks for one project. Ask for the tasks you need, not the catalog: use names, name_prefix, or query with detail: compact.
Input schemas missing for 11 of 15 tools. Only run_shell_command, start_shell_command, get_run_logs, search_run_logs, wait_run, and list_runs show explicit parameter schemas in visible code. Tools like list_projects, run_task, start_task, get_run, get_run_status, stop_run, and version_status have no documented input schema.
Output schemas not documented. No tool describes what fields it returns, their types, or structure. LLMs cannot plan downstream calls or extract required data (e.g., run_id from run_task result).
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2025-06-18+ | v2 |
Run a genuinely ad-hoc command, never a task withheld by a runner mode through this or another shell path. Prefer run_task/start_task, or a normal shell for output too large for a tail. A running receipt with promoted: true is normal: follow it; do not retry. Exactly one of command and block_id selects what runs; block_id comes from define_shell_block; working_directory must not accompany block_id. arguments is the full argv for an argv block, starting with the block's own argv. stdout_format=json returns stdout_json or stdout_json_error for completed output up to 65536 bytes.
Run a discovered task. Arguments are positional; tasks with declared parameters reject name=value. A running receipt with run_id and promoted: true is normal: follow it; do not start the task again.
Search stdout or stderr logs for a pattern.
Start a genuinely ad-hoc command asynchronously, never a task withheld by a runner mode through this or another shell path. Prefer run_task/start_task, or a normal shell for output too large for a tail. Exactly one of command and block_id selects what runs; block_id comes from define_shell_block; working_directory must not accompany block_id. arguments is the full argv for an argv block, starting with the block's own argv.
Start a discovered task asynchronously. Arguments are positional; tasks with declared parameters reject name=value. Prefer it for long check/verify gates. A running receipt with run_id is normal: follow it with wait_run or get_run_status; do not start the task again.
Stop a run owned by this server.
Check for updates.
Wait for a run; expiry does not stop it.
No tool annotations despite clear risk levels. Tools marked WRITE (run_task, start_task, run_shell_command, start_shell_command, stop_run) lack destructiveHint. READ_ONLY tools lack readOnlyHint. Agents cannot infer safety without annotations.
Parameter descriptions lack constraints. 'timeout' in wait_run has no range (0-65536?). 'limit' in list_runs has no bounds. 'pattern' in search_run_logs has no format guidance. LLMs cannot validate inputs without explicit constraints.
Error handling guidance absent. No tool describes what errors are retryable, what user-fixable errors look like, or how to recover. Descriptions like 'Run a discovered task' do not explain failure modes or next steps.