Loading ToolBench
A learning MCP server with inventory management tools and weather/climate integration
Three tools are registered with FastMCP decorator. Tool naming follows verb_noun pattern (listar, vender, obter) and is action-oriented. However, descriptions are brief and lack actionable guidance for LLM selection. Two tools have input schemas defined; one is parameter-less. No input validation error messages guide recovery. Output is unstructured text rather than JSON. No documented output schema. Tool risk annotations (READ_ONLY, WRITE) are present but not machine-readable in code. The async weather tool demonstrates capability but lacks retry/timeout guidance.
Lista todos os produtos do estoque com preços e quantidades.
Consulta API externa para ver o clima atual (Async).
Registra uma venda e abate do estoque no banco de dados.
Output schema not documented. Tools return unstructured text strings rather than typed JSON objects. LLMs cannot reliably parse results or chain tool calls. listar_produtos returns a formatted table string; vender_produto returns a status message; obter_previsao returns a sentence. No field definitions for downstream consumption.
Description length and specificity. listar_produtos: 'Lista todos os produtos do estoque com preços e quantidades.' (73 chars), lacks WHEN to call it or what structure it returns. vender_produto: 'Registra uma venda e abate do estoque no banco de dados.' (61 chars), does not mention side effects or prerequisites. obter_previsao: 'Consulta API externa para ver o clima atual (Async).' (53 chars), does not explain what city parameter does, error handling, or required environment variables (GEO_URL, WEATHER_URL). Baseline target: 194 chars avg.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 33 | - | v1 |
Parameter descriptions are minimal. vender_produto 'nome_exato' has description 'Exact product name to sell' (25 chars) but does not specify case sensitivity, what happens if multiple products match, or how to discover valid product names. LLM must infer from context. obter_previsao 'cidade' has no description in code, LLM does not know if it expects 'São Paulo', 'Sao Paulo', or an ID.
Error handling lacks recovery guidance. vender_produto: 'Erro: Produto ... não encontrado.' gives no hint about discovery. A 'product not found' response should suggest 'Call listar_produtos() to see available products.' obter_previsao catches all exceptions as 'Erro na conexão', does not distinguish between missing city, API timeout, or network failure. LLM cannot decide whether to retry, ask the user, or escalate.
No pagination or result limits declared. listar_produtos returns ALL products as a formatted table. If 10,000+ items exist, response bloats context window. Baseline requires declaring max results (20-50 recommended) and offering pagination (limit, offset, next_cursor).
No input validation or constraint documentation. vender_produto validates 'quantidade > 0' inside the function but returns a soft error message. No JSON Schema constraints (minimum: 1) prevent invalid input at the protocol level. obter_previsao has no length limit or format constraint on 'cidade', LLM could pass an empty string or 5000 chars.
Confirmation/dry-run missing for destructive tool. vender_produto modifies database (WRITE risk). No dry-run or user confirmation step. Agent could accidentally sell 1000 units of the wrong product without warning. Production pattern recommends confirmation_request for irreversible operations.
Hardcoded API credentials in environment variables (GEO_URL, WEATHER_URL) are passed to tool but not validated. If URLs are malformed or endpoints become unavailable, obter_previsao fails with generic 'Erro na conexão'. No timeout declared, request could hang indefinitely.