Loading ToolBench
AI-powered post-disaster relief orchestration backend with MCP tool server for managing shifts, tasks, supplies, notes, and broadcasts.
ReliefAI defines 17 tools with consistent naming (verb_noun pattern) and basic schemas. All tools have descriptions and input parameters with types. However, descriptions are often generic (10-50 chars), parameter descriptions lack detail about constraints/formats, output schemas are undocumented, and error handling guidance is absent. No tool annotations (readOnlyHint/destructiveHint). Composition is reasonable, tools are single-purpose, but parameter design could better match chat data model (e.g., volunteer_id as integer-only vs accepting names). Average tool score: 62/100.
Assign a volunteer to a disaster.
Cancel a scheduled shift and record the reason.
Create a new broadcast message for a disaster.
Create a new note for a disaster.
Create a new supply item for a disaster.
Create a new task for a disaster response operation.
Detect blocks in the coming days where too few volunteers are scheduled. Returns the scan parameters alongside the gaps themselves: {window_days, block_hours, min_volunteers, gap_count, gaps}. Quote `window_days` when describing the period examined — say "the next 7 days" only because the payload says 7, never from memory. The window is configuration and can change.
Output schemas undocumented. No tool returns a documented structure, LLMs cannot predict what fields to extract or which IDs to use for chaining calls. E.g., does get_shifts return shift_id, volunteer_id, and role for downstream calls?
Parameter descriptions lack constraint details. E.g., 'role' in schedule_shift has no enum or format hint; 'priority' in create_task lacks valid values (low|medium|high|urgent?); 'channel' in create_broadcast lists options in description but not as enum.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). LLMs cannot distinguish safe reads from destructive writes without explicit hints. E.g., cancel_shift and update_supply are destructive but unmarked.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2025-06-18+ | v2 |
List all broadcasts for a disaster.
List all notes for a disaster, optionally filtered by tag.
List all shifts for a disaster, optionally filtered by date (YYYY-MM-DD).
List all supplies for a disaster, with critical status indicators.
List all volunteers for a disaster.
Get weather forecast for a disaster location.
List all tasks for a disaster, optionally filtered by status.
Create a new shift for a volunteer on a disaster response.
Update the quantity available for a supply item.
Update the status of a task, optionally appending notes.
Generic descriptions under 60 chars. E.g., 'List all supplies for a disaster, with critical status indicators' (60 chars) does not explain when to call it vs get_supplies, what 'critical' means, or what fields are returned.
No error handling guidance. Tools do not document what errors are retryable, user-fixable, or fatal. E.g., if assign_volunteer fails because volunteer_id is invalid, the LLM has no recovery path.