Loading ToolBench
A C++23 agentic AI runtime that connects to MCP servers, discovers tools, and orchestrates a ReAct conversation loop with an LLM. Includes a shell command execution MCP server.
Single tool 'run_command' has a clear verb-noun name and adequate schema with typed parameters. However, the description (95 chars) is below the 194-char baseline and lacks critical safety/consequence information. No parameter descriptions exist in the schema, only inline in code comments. Output schema is undocumented. The tool is destructive (executes arbitrary shell commands) but lacks error handling guidance, confirmation patterns, or permission gates. Security is partially addressed via denylist and environment flag, but the tool remains high-risk without proper LLM-facing safeguards.
Executes an arbitrary shell command locally and returns combined stdout/stderr.
Tool description lacks consequence clarity. Does not state that this tool MODIFIES LOCAL STATE and executes arbitrary commands. LLMs cannot determine safety/reversibility without explicit 'destructive' or 'irreversible' language.
Input parameters 'command' and 'input' lack descriptions in the JSON schema. Schema only has type and inline code comments. LLMs cannot read code comments, they need descriptions in the schema itself.
No output schema documented. Tool returns {'output': string} but LLMs do not know the structure, max length (8000 chars), or truncation behavior without explicit schema.
Error handling does not guide recovery. Errors like 'Command rejected: matches a denylisted catastrophic pattern' are clear, but missing/invalid command errors lack actionable guidance. No suggestion of alternatives or next steps.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | <=2025-11-25 | v2 |
No confirmation or dry-run pattern for destructive operations. Agents can execute 'rm -rf' variants (not in denylist) without user confirmation, risking data loss.