Loading ToolBench
CLI for ITA Matrix airfare search (reverse-engineered JSON API) with MCP server support for flight search, date scanning, location lookup, and airline resolution
Five tools with complete input schemas, descriptions, and type definitions. All parameters have descriptions and constraints (min/max, enums via Field descriptions). Tool annotations present (readOnlyHint, idempotentHint). Naming follows verb_noun pattern (search_*, show_*). Key gaps: output schemas not documented in code; no error handling guidance; descriptions are functional but brief (avg ~80 chars, below 194-char baseline); no pagination guidance for large result sets; no recovery hints for common failures.
Resolve airline names and codes.
Scan a departure-date range and return the cheapest date results.
Search ITA Matrix for one-way or round-trip flights.
Autocomplete Matrix city and airport locations.
Search and expand one Matrix solution with segment, aircraft, and RBD details.
Output schemas not documented. Tool descriptions state what is returned (e.g., 'Search ITA Matrix for flights') but do not specify the response structure, field names, or types. LLMs cannot plan downstream tool calls or extract specific fields without knowing the response schema.
No error handling guidance. Tools lack descriptions of failure modes, recovery steps, or actionable error messages. E.g., if search_flights fails due to invalid IATA code, the tool should guide the LLM to call search_locations first.
Pagination not addressed. search_flights and search_dates accept 'limit' parameters but do not document whether results are paginated, how to fetch the next page, or what happens when results exceed the limit. Large result sets could exhaust context.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2026-07-28+ | v2 |
Tool descriptions are brief (avg ~80 chars vs. 194-char baseline). Descriptions state WHAT the tool does but lack WHEN to use it or dependencies. E.g., search_flights does not hint that search_locations should be called first if the user provides a city name instead of IATA code.
Parameter constraints in descriptions use prose instead of formal enums. E.g., 'COACH, PREMIUM_COACH, BUSINESS, or FIRST' is stated in the description but not as a JSON Schema enum. LLMs may hallucinate invalid cabin values.