Loading ToolBench
Multi-server MCP architecture for climate data analysis, including VIIRS fire detection and Aqueduct water resource data with tool calling integration
11 tools with mostly complete schemas and descriptions, but significant gaps in parameter documentation and error handling. Tool names follow verb_noun convention (list_, get_, query_, count_, describe_, summarize_). Descriptions range 50-250 chars, meeting baseline minimums. However, many parameters lack descriptions or constraints. No error recovery guidance, no output schema documentation, and no tool annotations (readOnlyHint/destructiveHint). The 'filters' parameter in query_viirs_data uses additionalProperties:true, which is vague and invites hallucinated filter names. Missing guidance on when to use similar tools (e.g., query_viirs_data vs query_recent_fires vs query_fires_by_location). No pagination or result-limiting strategy documented despite tools returning potentially large datasets.
Count fires near a location without fetching all records. Use this for 'how many fires' queries to get exact counts.
Count fire detections on a specific date (YYYY-MM-DD format) using acquisition_timestamp. Use this for queries like: 'how many fires on October 20, 2025'.
Return metadata and documentation URIs for the VIIRS Thermal Fire Dataset.
Get information about the VIIRS thermal records table
Get summary statistics for VIIRS thermal data (fire counts, temperature stats, etc.)
List all available VIIRS DuckDB tables
query_viirs_data uses 'filters' with additionalProperties:true, allowing arbitrary filter names. LLMs will hallucinate invalid filter keys (e.g., 'fire_intensity', 'temperature_range') that the backend doesn't support. Replace with explicit enum or document valid filter keys.
No output schema documentation for any tool. LLMs cannot plan downstream calls or extract fields without knowing response structure. Document what fields each tool returns (e.g., query_viirs_data returns [{'latitude', 'longitude', 'confidence', 'timestamp', ...}]).
No error handling or recovery guidance. Tools return raw database errors without actionable next steps. Add error classification (retryable vs user-fixable) and suggest alternatives (e.g., 'Date not found. Try query_recent_fires() instead.').
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | <=2025-11-25 | v2 |
Query fires within a geographic bounding box (latitude/longitude coordinates)
Get recent fire detections within the last N hours. Returns location, intensity, and confidence data.
Query VIIRS thermal hotspot data with filters (date range, location, confidence, etc.)
Change the VIIRS DuckDB database path and reconnect
WHEN TO USE: User asks for documentation summary, technical details about VIIRS sensors, or methodological information. DO NOT USE for fire data queries. EXAMPLES: 'Summarize the documentation', 'How does VIIRS detect fires?', 'What are the technical specifications?'. RETURNS: Key sections and headings from the local documentation file.
Three similar query tools (query_viirs_data, query_recent_fires, query_fires_by_location) lack clear disambiguation. Descriptions say WHEN to use each, but LLMs still conflate them. Add explicit 'Use this instead of X when Y' guidance in each description.
No pagination or result-limiting strategy documented. Tools accept 'limit' but no guidance on max values, default behavior, or how to fetch next page. Large result sets will exhaust context windows.