Loading ToolBench
MCP server for querying Brazilian Bank (Banco do Brasil) account statements and transaction data
Single tool 'consultar_extrato' has a basic description but lacks critical quality markers. Input schema is present with 4 parameters, all with type and description, but output schema is completely undocumented. The tool description (58 chars) is below the 194-char baseline and does not explain WHEN to use it, WHAT it returns, or dependencies. No error handling guidance, no pagination documentation despite the code implementing automatic pagination. No tool annotations (readOnlyHint, idempotentHint). The implementation shows sophisticated pagination logic and data enrichment, but none of this is surfaced in the tool definition for LLM reasoning.
Busca o extrato com paginação automática (Loop de Elite).
Output schema completely undocumented. Code returns a dict with 'resumo' (summary with totals, timestamp) and 'detalhes' (transaction list), but LLM has no visibility into this structure. Forces LLM to guess field names and types.
Tool description (58 chars) is below 194-char baseline and lacks context. Does not explain: what data is returned, when to call this vs alternatives, prerequisites (valid agency/account required), or that pagination is automatic. LLM cannot determine if this is the right tool without trial.
No error handling guidance in tool definition. Code catches HTTPError (404 returns None, others log and break), but LLM receives no recovery hints. If account not found, LLM does not know to try search_account or validate inputs.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | <=2025-11-25 | v2 |
Date format constraint in parameter description is vague. States 'DD/MM/AAAA format' but does not specify: are leading zeros required? Is 'AAAA' a typo for 'YYYY'? Code strips leading zeros in _formatar_data_bb(), but LLM has no visibility into this transformation.
No tool annotations. Code is read-only (GET request, no state mutation) and idempotent (same inputs always return same results), but readOnlyHint and idempotentHint are absent. LLM cannot infer safety properties.