An autonomous auditor server for analyzing public spending and financial transparency data using RAG (Retrieval-Augmented Generation) with LangChain and MCP
Critical failures across all dimensions. Tool definitions lack proper schemas, descriptions are incomplete or duplicated, and parameter documentation is minimal. The server registers 3 tools but two are duplicates with conflicting schemas (guardar_relatorio appears twice with different parameter names: 'nome_arquivo' vs 'nome_fornecido'). No input validation, error handling guidance, or output schema documentation. Naming is acceptable (verb_noun pattern) but insufficient to overcome structural deficiencies.
Busca informações financeiras e gastos públicos no banco de dados. Use esta ferramenta ANTES de responder qualquer pergunta sobre os dados.
Salva um relatório no disco local do computador. Parâmetros: - nome_arquivo: O nome do arquivo a ser salvo (ex: fraude_cafe.txt). IMPORTANTE: inclua a extensão .txt. - conteudo: O texto completo do relatório detalhando o problema.
A função dessa ferramenta é salvar o seu relatório no computador local.
Duplicate tool registration: 'guardar_relatorio' defined twice with conflicting parameter schemas (nome_arquivo vs nome_fornecido). LLM cannot disambiguate which to call.
Input schemas lack proper JSON Schema structure. 'buscar_dados_transparencia' has only a 'query' string with minimal description. No type constraints, enums, or validation rules visible.
No output schema documentation. Tools return unstructured responses (e.g., 'SUCESSO: Arquivo...' string). LLM cannot plan downstream calls or extract structured data.
Missing error handling guidance. guardar_relatorio writes to disk without validation, permission checks, or recovery instructions. No guidance on what to do if write fails.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
Parameter descriptions are incomplete. 'nome_arquivo' description includes example values ('fraude_cafe.txt') which LLMs tend to reuse literally. No format constraints or validation rules.