Loading ToolBench
AgentCLI — call MCP servers and OpenAPI APIs from the command line. One CLI, any tool backend.
AgentCLI's echo-server fixture defines 5 tools with complete input schemas and basic descriptions. However, descriptions are minimal (10-60 chars, well below the 194-char baseline), lack actionable context for LLM selection, and parameter descriptions are sparse or absent. Tool names are generic (echo, complex, fail, slow, double) and do not follow verb_noun conventions. No output schemas are documented. Error handling is minimal, the 'fail' tool returns a bare error string with no recovery guidance. The server is STDIO-only, which is a hard transport limitation. Overall, this is a test fixture, not production-grade tooling.
Takes object parameters that cannot be flags
Returns a doubly-encoded JSON payload (as many HTTP servers do)
Echo a message back, optionally repeated and uppercased
Always reports a tool error
Sleeps before responding
Tool names lack action verbs and clarity. 'echo', 'complex', 'fail', 'slow', 'double' do not follow verb_noun convention (e.g., 'repeat_message', 'validate_spec', 'report_error'). LLMs cannot infer intent from these names alone.
Descriptions are too brief (10 - 60 chars vs. 194-char baseline). 'Echo a message back, optionally repeated and uppercased' lacks context on WHEN to use this tool or what it returns. No guidance on parameter relationships (e.g., 'upper' and 'mode' both uppercase, why both?).
Parameter descriptions are missing or minimal. 'times' has a description, but 'upper' and 'mode' lack clarity on their interaction. 'spec' in 'complex' is described as 'Arbitrary spec object', too vague for LLM reasoning.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | 2025-06-18+ | v2 |
No output schemas documented. Callers cannot predict response structure. 'echo' returns text, 'complex' returns JSON, but this is inferred from code, not declared in tool metadata.
Error handling is non-actionable. 'fail' tool returns 'boom: intentional failure' with no recovery guidance. 'unknown tool' error gives no hint on valid tool names. LLMs cannot self-correct.