Loading ToolBench
An agentic flood risk assessment and emergency response coordination system for the Mbagathi Basin in Nairobi using Google ADK, Vertex AI, and Cloud Spanner.
FloodPulse Nairobi exhibits significant definition quality gaps across all 15 tools. While tool names follow verb_noun conventions (e.g., update_mission_context, fetch_basemap), descriptions are inconsistent, some are adequate (10-50 chars) but many lack actionable detail. Critical issue: input schemas are partially visible but incomplete. The ToolContext parameter appears in 5 tools (update_mission_context, generate_active_mission_map, analyze_terrain_tool, get_active_weather_pulse, generate_explorer_avatar) with only a generic description 'ToolContext from google.adk.tools' or similar, providing no guidance on what fields it contains or how to populate it. Parameters like 'config' in generate_explorer_avatar lack type details. Output schemas are not documented for any tool, LLMs cannot predict return structures. Error handling is minimal; most tools raise generic exceptions without recovery guidance. No tool declares permissions, security scopes, or idempotency guarantees. The codebase shows partial implementations (weather_tools.py truncated mid-function), making full assessment impossible.
Adapter that retrieves responder info from session and performs structured terrain analysis using vision AI on mission maps.
Calculates the safest route between a responder and an at-risk resident using network graph weights to avoid flooded/high-risk paths.
Orchestrates the dispatch: finds the nearest responder, calculates safe routes, updates node status, and logs the dispatch event.
Fetches satellite tile from Google Maps.
Closes the rescue mission: marks resident as safe, responder as 'Clear', and logs completion event.
Adapter function that uses ToolContext to pull mission data from session state and generates a mission map.
ToolContext parameter lacks schema detail. Five tools (update_mission_context, generate_active_mission_map, analyze_terrain_tool, get_active_weather_pulse, generate_explorer_avatar) accept ToolContext with only generic description. LLMs cannot determine what fields to populate or how to construct this object.
No output schemas documented. Tools return dicts, strings, or file paths with no declared structure. LLMs cannot plan downstream calls or extract required fields (e.g., what fields does analyze_terrain_tool return? What is the structure of get_available_responders response?).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | 2026-07-28+ | v2 |
Generates consistent portrait and icon using multi-turn Gemini chat session for character consistency.
Generates a mission map by fetching a basemap, overlaying responder icon, and saving the result.
Adapter that uses injected ToolContext to retrieve mission coordinates from session and calls the core weather tool.
Queries the database to retrieve all responder nodes that are currently in 'Moderate Pulse' or 'Critical Pulse' status.
Fetches real-time weather data or generates a simulation pulse for rainfall and flash flood risk assessment.
Uses index-based proximity logic to find the closest available responder to a resident.
Queries Spanner to get the full telemetry for all nodes in the basin.
Queries Spanner and returns all matching nodes for a given name.
Saves user-provided mission details into the persistent session state.
Minimal error handling and no recovery guidance. Tools raise generic exceptions (ValueError, FileNotFoundError, ConnectionError) without actionable next steps. E.g., analyze_terrain_tool raises FileNotFoundError if map missing, should suggest calling generate_mission_map first.
Destructive operations (dispatch_to_resident, finalize_rescue) lack confirmation or dry-run support. Agents can irreversibly update node status without verification, risking incorrect dispatch or rescue closure.
Parameter descriptions lack constraints and format guidance. E.g., 'zoom' parameter in fetch_basemap and generate_mission_map has no min/max bounds (valid range unknown). 'status_code' in finalize_rescue has no enum or valid values listed.