Loading ToolBench
Flight-hacker's scanner for short trips: split tickets, trip-shape search across a month, and it knows what's on when you land.
Three tools with complete input schemas and descriptions. Tool names follow verb_noun pattern (splitfare_find_flights, splitfare_scan_month, splitfare_whats_on). Descriptions are detailed (150-250 chars) and explain WHAT, WHEN, and constraints. All parameters have types and descriptions. However, output schemas are not formally documented in the code, only inferred from return statements. Error handling is present (ToolError class, validation checks) but error messages lack recovery guidance. No parameter enums despite constrained inputs (e.g., out_dow could be an enum). Missing documentation of output structure for LLM planning.
Price one out/back date pair (YYYY-MM-DD): direct fares plus split-ticket self-transfer combos via the configured hub airports, ranked by total price. arrive_by / home_by are optional HH:MM caps on the outbound / return arrival time. top limits distinct routings returned per direction. Takes ~1.5s per uncached route-date (up to ~1 min cold for many hubs).
Sweep a month (YYYY-MM) for the cheapest N-night windows. out_dow limits outbound weekdays (e.g. 'thu,fri,sat,sun'). deep = how many candidate windows get full split-ticket pricing (0 = all — thorough but SLOW on a cold cache, potentially 10+ minutes; 4-8 is a good default). Windows that miss the time caps are returned under 'excluded' with the reason and what relaxing the cap would cost.
Destination event calendar via the user's configured events provider (e.g. Resident Advisor for their city) for a date, optionally up to 7 consecutive days. Returns events with venue, lineup, times and, where the provider has them, door prices.
Output schemas not formally documented. Return types are inferred from code (dicts with 'adults', 'currency', 'outbound', 'return', etc.) but not declared in tool metadata. LLMs cannot plan downstream operations without knowing response structure.
Parameter constraints not expressed as enums. 'out_dow' accepts 'thu,fri,sat,sun' but is a free-form string, LLMs may hallucinate invalid day names. 'days' (1-7) and 'nights' (1-14) lack min/max constraints in schema.
Error messages lack recovery guidance. ToolError exceptions (e.g., 'invalid out_date') tell the LLM what failed but not what to do next. Should suggest 'Use YYYY-MM-DD format' or 'Call splitfare_find_flights with a valid date'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Tool descriptions include performance caveats ('~1.5s per uncached route', '10+ minutes cold') but no guidance on when to use each tool. LLMs may not understand that splitfare_find_flights is for specific dates while splitfare_scan_month is for exploration.
splitfare_whats_on description is truncated in source ('Destination event calenda', missing 'r'). Full description not visible; cannot assess completeness.