Loading ToolBench
Workflow-continuity bridge for AI coding agents. Supervise, approve, and redirect Claude Code, Copilot, Cursor, and other agents from any device — async. Includes VS Code extension, JetBrains plugin, and codeam-cli.
Six tools with mixed quality. All have descriptions (good), but schemas are incomplete or inferred. Tool names follow verb_noun pattern (ensureGhCli, ensureGhAuth, ensureGlabCli, readLocalMcpServers, mergeWithLocalMcpServers, createToolCallWatchdog). Descriptions range 80 - 250 chars, meeting baseline. However, input schemas show only high-level object types without nested field definitions, parameter descriptions, or type constraints. No output schemas documented. Error handling guidance absent. Tools 4 - 6 appear inferred from file paths rather than explicit registration code.
Creates a per-`tools/call` watchdog for an MCP relay. A wrapped/remote MCP server can accept a `tools/call` and NEVER send a response (a hung remote or a stdio child that only answers while some external process is up). With no per-request timeout the agent's turn wedges FOREVER. This watchdog bounds every `tools/call`: if the server doesn't answer within `timeoutMs`, it synthesizes a JSON-RPC error for that request id so the agent unblocks with a clean tool error, and then DROPS any late reply so the agent never sees two responses for one id.
Authenticate `gh` with the user's token — but ONLY if it isn't already logged in, so we never clobber an existing GitHub account the user set up on their own box. Best-effort, never throws.
Ensure `gh` (GitHub CLI) is available, returning the command to invoke it (`'gh'` if it was already on PATH, or the absolute path to the binary we installed), or null if it could not be made available. Downloads and installs the official static binary from GitHub releases if not present.
Install the GitLab CLI (`glab`) so the agent can drive GitLab MRs (`glab mr ...`) the same way the GitHub deploy gets `gh`. Mirrors ensureGhCli: static release binary, no root, PATH-prepend. Returns the resolved command or null (never throws; git push/pull already work via the credential helper).
The single merge policy shared by every composition point that builds the final `mcpServers` list for a session: Agent Toolkit integration servers first, then box-local servers appended — on a `name` collision the integration wins (it's per-user, backend-managed) and the local duplicate is dropped with a warning rather than silently shadowing it.
Input schemas lack nested field definitions and parameter descriptions. All six tools show only top-level object types (e.g., 'runner': {type: object}) without documenting what fields runner contains, their types, or constraints.
No output schemas documented for any tool. LLMs cannot infer what fields to expect in responses, breaking downstream tool chaining and forcing agents to guess at response structure.
No error handling guidance. Tools like ensureGhCli return 'null if it could not be made available' but do not explain what the agent should do next (retry, ask user, abort). Descriptions lack recovery paths.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
Reads `~/.codeam/mcp-servers.json` and returns the ACP `McpServer[]` spec for each valid entry. Missing file → `[]` (the common case — no local config). Malformed JSON / non-array top level / an entry that fails validation → warned once per call and skipped; this function never throws.
Parameter 'runner' and 'deps' are injectable test doubles, not user-facing inputs. Exposing test infrastructure as tool parameters violates separation of concerns and confuses LLMs about what they control.
Tools 4 - 6 (readLocalMcpServers, mergeWithLocalMcpServers, createToolCallWatchdog) are inferred from file paths and descriptions only; no explicit tool registration code visible. Cannot verify actual schema or parameter binding.