Loading ToolBench
A multi-agent desktop automation system that delegates tasks to specialized AI agents (code generation, document parsing, complex reasoning, and computer use) via an agentic loop with Anthropic's Claude models.
This server exhibits severe quality gaps across naming, descriptions, schemas, and error handling. Most tools use vague, non-action-oriented names ('code_agent', 'complex_agent', 'router_agent') that obscure intent. Descriptions are minimal and non-functional, they lack context about WHEN to use each tool, WHAT to expect back, and HOW errors should be handled. Parameter descriptions exist but are sparse (e.g., 'The code request message' is 25 chars, acceptable but generic). Critically, output schemas are completely undocumented, the LLM has no idea what shape of data each tool returns. Error handling is present but generic ('Errore durante la chiamata a Code Agent') with no recovery guidance. The 'computer_use' tool is dangerous: it accepts free-form 'action' strings (bash commands, mouse operations) with no input validation, no rate limiting, and no per-action safety gates. Tool composition is problematic, tools invoke other HTTP endpoints but do not document dependencies or failure modes. No tool declares required permissions, scope, or idempotency guarantees. Italian descriptions mixed with English parameter docs reduce clarity for international teams.
A tool that allows the agent to run bash commands.
Invia un messaggio all'agente specializzato nella scrittura di codice.
Risponde a domande complesse in linguaggio naturale.
A tool that allows the agent to interact with the screen, keyboard, and mouse of the current computer.
Esegue operazioni sul computer virtuale usando gli strumenti base come bash, computer e editor.
Inoltra una richiesta all'agente che analizza documentazione tecnica per fornire indicazioni su codice e API.
Decide quale agente delegato usare in base al messaggio utente.
Tool names do not start with action verbs and obscure intent. Names like 'code_agent', 'complex_agent', 'router_agent' fail the verb_noun convention. LLMs cannot infer what these tools do from the name alone, 'code_agent' could mean 'get code suggestions', 'run code', 'parse code', or 'send message to agent'. Contrast with 'generate_code', 'answer_question', 'route_request', which are unambiguous.
No output schemas documented. The LLM does not know what fields to expect from any tool's response. Code samples show tools return a ToolResult with 'output' or 'error' fields, but downstream tools that depend on specific return values (e.g., IDs, references) cannot be chained safely. This forces the LLM to guess what shape of data it gets back.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 34 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 21 | - | v1 |
Description text is Italian mixed with English, creating ambiguity for multilingual teams. More critically, descriptions lack the WHEN/WHY context. E.g., 'Invia un messaggio all'agente specializzato nella scrittura di codice' says WHAT but not WHEN to use it instead of 'complex_agent' or 'doc_parser_agent'. No explanation of dependencies (does this tool depend on prior context?) or prerequisites.
Error messages are generic and non-actionable. E.g., 'Errore durante la chiamata a Code Agent: {e}' tells the LLM only that the call failed, not why, whether it is retryable, or what to do next. Per the rubric, errors must categorize as retryable/user-fixable/fatal and suggest recovery steps.
The 'computer' tool accepts free-form 'action' strings (bash, mouse clicks, keyboard input) with no input validation, no permission gates, and no rate limiting. This is a critical security and safety gap. An attacker or misbehaving agent could invoke 'bash command rm -rf /' without friction. No confirmation pattern, no dry-run support.
No tool declares required permissions, scope, or idempotency guarantees. The LLM does not know which tools are safe to retry or whether a tool read-only or writes state. 'bash' and 'computer' can both execute destructive commands, they should declare 'write' scope and require confirmation for irreversible actions.
Tool composition is fragile. 'code_agent', 'complex_agent', 'doc_parser_agent', and 'router_agent' all invoke HTTP endpoints on localhost:8000, but there is no documentation of what those endpoints return, how they fail, or what happens if the server is down. The LLM cannot reason about dependencies or fallback strategies.
Parameter 'model' appears in code_agent, complex_agent, doc_parser_agent with no description, no enum constraint, no default. LLMs will hallucinate invalid model names. Should enumerate allowed models (claude-3-sonnet-20250514, gpt-4, etc.) and provide a sensible default.
The 'bash' tool has no documentation of which commands are allowed vs. blocked. An LLM could invoke 'rm -rf /' without friction. No examples of valid usage. No timeout declared for hanging commands. No explanation of whether output is captured and returned or only logged.
The 'computer' tool's 'action' parameter lacks validation. The description lists 16 possible actions (key, type, mouse_move, left_click, ...) as free text, not as an enum. LLMs will invent actions like 'screenshot_and_analyze' or 'move_and_click_simultaneously' that do not exist, causing silent failures.