Loading ToolBench
MCP server that exposes build error and output capture tools for AI coding assistants. Drains pending build messages from local and remote sources.
devtap exposes 2 tools with acceptable naming conventions but significant gaps in parameter documentation, schema completeness, and error guidance. Tool names follow verb_noun pattern (get_build_errors, get_build_status), which is correct. However, both tools accept zero parameters (empty properties map) and lack descriptions for their outputs. The tool descriptions are present and reasonably detailed (108 and 78 chars respectively, within the 10-1024 range), but parameter documentation is absent since neither tool accepts input parameters. No output schemas are documented, LLMs cannot predict what fields to expect in responses. Error handling exists (IsError flag on CallToolResult) but lacks recovery guidance. The server registers tools explicitly in toolDefinitions() with proper structure, but the schemas are minimal and output is underdocumented.
Get pending build errors and output captured by devtap. Call this at the start of each task and before writing or editing code to check for build failures that need fixing. A separate terminal may have captured new build errors or user messages at any time. If build succeeded, acknowledge briefly without repeating the output. If build failed, present the error output verbatim — do not summarize or reinterpret error content.
Get a summary of pending build output counts for the current session.
Output schemas are not documented. LLMs cannot predict the structure of responses from get_build_errors or get_build_status, forcing them to infer fields and risk misuse of returned data.
No per-tool error guidance in descriptions. Error responses (CallToolResult with IsError=true) are generated at runtime but the tool descriptions do not explain what errors are possible, when to retry, or how to recover. For example, get_build_errors may fail to drain from a source, but the description does not mention this or advise fallback actions.
Tool descriptions lack 'when to use' context. get_build_errors says 'Call this at the start of each task' which is helpful, but get_build_status offers no guidance on when it should be invoked vs get_build_errors, forcing LLMs to reason about the distinction.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 33 | 2024-11-05+ | v1 |
Tool descriptions include directive language ('Call this at the start') that conflates instructions with descriptions. Tool descriptions should describe WHAT the tool does and WHY, not HOW or WHEN an agent should use it. This is agent-facing intent guidance, not tool documentation.