Loading ToolBench
MCP server for American Airlines - let AI agents search flights, manage bookings, check in, and access AAdvantage rewards on aa.com
Server has 17 tools with explicit schemas and descriptions. Naming follows verb_noun convention (search_flights, get_flight_details, select_seats). Most descriptions are adequate (100-250 chars), explaining WHAT and WHEN. However, critical gaps: (1) output schemas are completely undocumented, no tool declares what fields it returns, forcing LLMs to guess; (2) error handling is absent, no guidance on retryability, user-fixable vs fatal errors, or recovery paths; (3) several parameters lack type constraints (e.g., flightId is string but no format/pattern guidance); (4) destructive operations (checkout, cancel_trip) have confirm flags but no dry-run/preview mechanism documented in schema; (5) no parameter descriptions for status/login/logout tools (empty properties). Composition is reasonable, tools chain logically (search → select → checkout), but missing IDs in responses would break chains. Average tool score: 62/100.
Add checked baggage to the booking. AA charges per bag per segment.
Cancel an American Airlines reservation. IMPORTANT: Set confirm=true only after explicit user confirmation. Without confirm=true returns a preview.
Complete online check-in for an American Airlines flight. Requires a valid reservation and returns boarding pass information.
Complete the American Airlines booking. IMPORTANT: Set confirm=true only after getting explicit user confirmation. Without confirm=true returns a preview.
Retrieve AAdvantage account information including miles balance, elite status, tier progress, and upcoming milestone expirations.
Retrieve the boarding pass for a checked-in flight. Returns barcode, seat, gate, boarding time, and other details.
No output schemas documented. Tools return data but LLMs cannot see what fields to expect. Forces guessing on field names, types, and availability. Breaks tool chaining (e.g., search_flights returns flightId but no schema confirms this).
Empty input schemas for status, login, logout, get_aadvantage. These tools accept no parameters but schema is {properties: {}} with no description. Violates pattern:tool-description, every parameter needs a description.
No error handling guidance. Tools like checkout and cancel_trip are irreversible but provide no recovery hints, retryability classification, or user-fixable error messages. LLM cannot distinguish 'retry this' from 'ask user' from 'fatal'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
Get detailed information about a specific flight from search results, including connection info, aircraft type, baggage policy, and fare rules.
Look up an existing American Airlines reservation by record locator (confirmation code).
View the seat map for the selected flight showing available and occupied seats, seat types (Main Cabin Extra, Preferred, First), and fees.
Initiate American Airlines login flow. Opens aa.com so the user can log in manually with their AAdvantage credentials. After logging in, call status to verify and save the session.
Clear saved American Airlines session and cookies. Use this to log out or reset authentication state.
Change flights on an existing American Airlines reservation. Initiates the flight change flow — use search_flights to find new options.
Search for flights on American Airlines (aa.com). Returns flight options with flight numbers, times, stops, aircraft, and fares.
Select an outbound (and optionally a return) flight after searching. This adds the flight to your booking session.
Choose seats for passengers on the selected flight. Use get_seat_map first to see available seats.
Check American Airlines login status and AAdvantage session info. Use this to verify authentication before performing other actions.
Request a cabin upgrade for an existing reservation using AAdvantage miles or cash.
Destructive operations lack dry-run/preview mechanism in schema. checkout and cancel_trip have confirm flags but no documented preview behavior. Agents cannot safely test before committing.
Parameter constraints missing. flightId, recordLocator, and seat numbers lack format/pattern guidance. LLMs may pass invalid values (e.g., 'flight-123' instead of '0'). No min/max for numeric params (bags, adults, children, maxResults).