Loading ToolBench
MCP server for interacting with AirTrail flight management system
AirTrail MCP exhibits significant definition quality gaps across all three tools. While tool names are action-oriented (list_, get_, add_), descriptions lack depth and parameter documentation is inconsistent. The add_flight tool has a critical schema inconsistency: the input schema shows 'departure_airport_icao_code' and 'arrival_airport_icao_code', but the FlightModel uses aliased fields ('from_' with alias 'from', and 'to'). Schemas are partially visible but incomplete in the provided source. Error handling is minimal, tools return raw strings or None without recovery guidance. No output schemas are documented. The list_flights tool accepts no parameters despite needing pagination support for large result sets.
Add a new flight to Airtrail
Retrieve flight itinerary information including departure and arrival ICAO airport codes for a specific flight number. Returns airport codes in ICAO format.
List all flights from Airtrail
list_flights lacks pagination (limit, offset, page_size parameters). Without pagination controls, large result sets blow the context window and LLMs cannot handle incremental discovery.
add_flight has a critical schema mismatch: the input schema parameter names ('departure_airport_icao_code', 'arrival_airport_icao_code') do not match the FlightModel field names ('from_' aliased to 'from', 'to'). This will cause validation errors or silent failures when LLMs pass parameters matching the published schema.
No output schemas documented for any tool. LLMs cannot predict what fields to expect, forcing them to parse responses blindly and guess at downstream field names for chaining calls.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 45 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Tool descriptions are under 100 characters and lack actionable context. They do not explain WHEN to use each tool, what prerequisites exist, or what the return value structure is. E.g., 'List all flights from Airtrail' does not tell LLMs whether this returns paginated results, what fields are included, or how many results to expect.
Error handling returns raw strings (e.g., 'Request failed with status X', 'Response is not valid JSON') without recovery guidance. LLMs receive no hint about whether to retry, ask the user, or give up.
add_flight's 'seats' parameter expects an array but no validation is documented. The SeatModel requires 'userId', but the parameter description does not clarify this requirement or whether guestName is an alternative. LLMs will pass incomplete seat objects.
API credentials (airtrail_api_key) are injected at runtime via environment variables (good), but the AirtrailClient is instantiated at server startup. If credentials are invalid or expire, the entire server fails silently with no runtime error reporting.