Loading ToolBench
A C++ Model Context Protocol (MCP) server implementation with developer assistance tools
This C++ MCP server defines 15 tools with C++ parameter structs and basic handler implementations. However, critical quality gaps prevent a higher score: (1) Tool descriptions are present but generic and lack LLM-optimization guidance. Most hover around 50-80 chars, acceptable baseline but not exceptional. (2) Input schemas are visible in .hpp structs (LoadConfigParams, ExecuteVSCodeTaskParams, etc.) with proper C++ typing, but NO JSON Schema output documentation is visible. The rubric requires documented output schemas for all tools; source only shows C++ return type 'std::string' with no structured field definitions. (3) Parameter descriptions in structs are minimal ('Path to workspace (optional, defaults to current directory)'), missing actionable format constraints, validation rules, or usage context. (4) Error handling is implemented as string returns (e.g., 'Sync target configuration not found') but lacks structured error codes, recovery guidance, or actionable next steps. (5) High-risk operations (docker_build, docker_run, sync_files, start_gdb, execute_workflow) lack confirmation patterns or dry-run capability despite their destructive/side-effect nature. (6) No visible tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite explicit risk markings in the server definition. (7) Resource composition is reasonable, tools chain via workspace_path and config names, but output schema design is opaque. Average tool score across 15 tools: 42/100.
Build a Docker image from configuration
Run a Docker container from configuration
Execute a VSCode task by name
Execute a named workflow by running its configured steps
Get project information including build systems, Docker configuration, VSCode setup, and dev assistant config
List available VSCode debug launch configurations from .vscode/launch.json
List available VSCode tasks from .vscode/tasks.json
List available workflows from configuration
No output schema documentation visible. Tools return C++ std::string with unstructured content (e.g., 'Configuration loaded successfully.\nProject: ...'). Rubric requires documented output schemas with typed fields so LLMs can parse results and chain tools. Pattern:response-shaper requires structured objects, not free-text.
Destructive/write operations lack confirmation patterns. docker_build, docker_run, sync_files (executes rsync), start_gdb, and execute_workflow modify state or trigger external processes but have no dry-run, explicit confirmation step, or user-guided elicitation. Agents can trigger destructive actions without safeguards.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 39 | <=2025-11-25 | v2 |
Load configuration from file or default location
Query dictionary entries from configuration by category and key
Query AI assistance prompts from configuration by category and key
Read file contents
Start remote GDB debugging session with configuration
Start a VSCode debug session with a named launch configuration
Sync files to a remote target using rsync
Error responses are unstructured strings without actionable recovery guidance. Example: sync_files returns 'Sync target configuration not found: ...' and lists available targets, which is good, but most tools return generic strings. No structured error classification (retryable vs user-fixable vs fatal), no error codes, no per-item success/failure for batch-like operations.
Tool annotations missing. Server definition marks Risk as READ_ONLY or WRITE but no readOnlyHint/destructiveHint/idempotentHint visible in schemas or handler code. Annotations help LLMs reason about side effects and retry safety. Current MCP spec (2026-07-28) recommends tool annotations.
Parameter descriptions lack validation rules and format constraints. E.g., 'config_path': 'Path to configuration file (optional)' doesn't specify format, allowed patterns, or validation. Rubric pattern:constrained-input requires explicit ranges, enums, regex, or character limits in descriptions so LLMs avoid invalid input.
List and query tools lack pagination or result limits. list_vscode_tasks, list_vscode_launches, query_prompts, query_dictionary, and list_workflows may return unbounded result sets with no mention of page/offset/limit or max result count. Rubric baseline: paginated-result pattern + mxe:enforce-result-limits (cap 20-50 items, state limit in description).
Tool descriptions are 50-80 characters on average, within acceptable baseline (p10=34, p90=392 in production) but lack LLM-optimization. Compare 'Get project information including build systems, Docker configuration, VSCode setup, and dev assistant config' (88 chars, somewhat generic) vs pattern:tool-description guidance: 'What does it do? When should I call it? What does it return?' Structure is vague; descriptions don't explain *why* an LLM would choose this tool over alternatives.
No visible idempotency guarantees. Operations like execute_workflow, execute_vscode_task, docker_build, and docker_run are non-deterministic or side-effect-heavy. If an agent retries on timeout, could these create duplicate deployments or run the same workflow twice? Pattern:idempotent-operation requires clarity on retry safety.
Possible command injection risk in sync_files. Code builds rsync command by string concatenation: cmd += ' --exclude \'' + excl + '\'' without shell escaping. If a sync target config includes malicious exclude patterns (e.g., with quotes or backticks), injection is possible. Pattern:tool-gateway requires input sanitization.