Loading ToolBench
Multi-agent AI system with specialized agents for real estate, news, weather, and finance. Uses MCP servers and external APIs. Built with pydantic-ai and Gemini models.
This MCP server exhibits significant definition quality gaps across all dimensions. While 7 tools are defined with basic schemas, descriptions are present but brevity and schema completeness issues dominate. Tool naming follows verb-first convention adequately (read_, writer_, get_), but parameter descriptions lack rigor. Most critically, no tool definitions are visible in the source code provided, only FastAPI/Streamlit integration code and Pydantic output schemas. The actual MCP tool registration, input validation, and error handling are not present in the supplied files, forcing assessment based on inference from the API and UI layers. This violates the 'must see actual tool definition' scoring rule, capping individual tool scores at 50 even where schemas exist in schemas/tools_schemas.py. Parameter descriptions in input schemas are extremely brief (5-10 chars) and lack actionable guidance. No error recovery hints, no permission gates, no audit logging visible.
Obtiene información sobre el valor del bitcoin en base a un país en formato ISO 4217
Obtiene noticias recientes de Colombia incluyendo títulos, descripciones, fechas y enlaces
Retorna la fecha y hora actual
Obtiene el tipo de cambio entre dos monedas (códigos ISO 4217)
Obtiene información climática de una ciudad colombiana incluyendo temperatura, viento, humedad, nubosidad, radiación UV y presión
Permite analizar archivos mediante URL pública (imagen, video, documento)
Permite generar un reporte de un tema en formato markdown con información de soporte que pueden venir de respuesta de otras tools
No visible MCP tool registration code in provided source. Tool definitions inferred from API/UI integration only. Cannot verify actual MCP server implementation, parameter constraints, or error handling patterns.
Parameter descriptions are critically brief (5 - 15 chars). Pydantic Field descriptions like 'Pregunta específica sobre el archivo' (22 chars) lack context on format, constraints, or when to use the tool. LLMs cannot determine valid inputs without detailed parameter guidance.
Tool names contain vague suffixes. 'get_bitcoint_tool' is misspelled and unclear (bitcoin? bitcoint?). 'get_trm_tool' uses domain jargon (TRM = 'tipo de cambio') without explanation. LLMs cannot infer purpose from these names.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 35 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 38 | - | v1 |
Parameter 'moneda_origen' and 'moneda_conversion' in get_trm_tool lack enum constraints. Description says 'ISO 4217' but does not state valid values (USD, COP, EUR, etc.). Free-form strings invite hallucinated currency codes.
No output schema documented for any tool. Pydantic classes (TimeNow, NewsItem, WeatherItem, etc.) exist in schemas/tools_schemas.py but are not linked to tool definitions or visible in MCP context. LLMs cannot plan downstream calls without knowing return field names and types.
No error handling guidance visible. Tools fail silently or return generic HTTP errors (e.g., HTTPException in main.py with 'Error al generar respuesta'). LLMs receive no recovery hints, retryability classification, or suggestions for self-correction.
Destructive tool 'writer_report_tool' (WRITE risk) has no confirmation or dry-run pattern. Tool writes markdown files to storage without agent verification. Risk of unintended content generation or overwrite.
Tool names in Spanish (e.g., 'read_media_tool' with Spanish parameter names 'tema', 'recursos', 'moneda_origen'). Mixed language naming creates friction for English-speaking LLM models and violates clarity conventions. Tool and param names should be in English for broad compatibility.
Tool 'get_news_tool' parameter 'limit' lacks bounds. Description states 'default 5, máximo 30' but JSON Schema does not enforce maximum. LLMs may request arbitrary limits, overwhelming the API.