Loading ToolBench
Boston real-time transit via the MBTA v3 API (api-v3.mbta.com)
Four well-named tools with clear, detailed descriptions (avg 180 chars) and complete JSON Schema input definitions. All parameters are typed and described. However, output schemas are not documented, LLMs cannot predict response structure for downstream chaining. No error handling guidance, no output field documentation, and no tool annotations (readOnlyHint). Parameter descriptions include examples (e.g., 'South Station', 'Red') which LLMs may reuse literally. Missing pagination documentation despite tools returning lists. Overall solid naming and input clarity, but incomplete for production agent use.
Active MBTA service alerts — Boston subway T delays, commuter rail disruptions, shuttle replacements, track changes, detours, elevator outages. Filter by route to check one line, e.g. is the Red Line delayed right now. Example: mbta_alerts({ route: "Red" })
Real-time MBTA train arrival and departure predictions at a Boston station — answers "when is the next train Boston", next Red Line subway at South Station, Green Line trolley, commuter rail departures, Silver Line and bus arrivals. Accepts a station name ("South Station", "Harvard") or a place id ("place-sstat"). Falls back to the published schedule when live predictions are empty (common for commuter rail off-peak). Example: mbta_departures({ stop: "South Station", route: "Red" })
List MBTA routes for the Boston subway T, commuter rail, bus, and ferry — route ids, names, and the destination each direction heads toward. type: 0-1 subway/light rail (Red, Orange, Blue, Green, Mattapan), 2 commuter rail, 3 bus (includes Silver Line), 4 ferry. Default returns subway + commuter rail + ferry; pass type 3 for buses. Example: mbta_routes({ type: "2" })
List the stops and stations on an MBTA route — Boston subway T lines, commuter rail lines, bus routes, ferries. Returns stop id, name, municipality, and wheelchair accessibility, in route order. Example: mbta_stops({ route: "Red" }) or mbta_stops({ route: "CR-Worcester" })
Output schemas not documented. LLMs cannot predict response structure (fields, types, nested objects) for downstream tool chaining or data extraction. Agents must guess what fields are returned.
Parameter descriptions include example values (e.g., 'South Station', 'Red', 'place-sstat'). LLMs tend to reuse examples literally rather than adapting to context, causing incorrect calls.
No error handling guidance. Tools lack recovery hints for common failures (rate limit, invalid station, API errors). Agents cannot self-correct on failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
No tool annotations. Tools lack readOnlyHint, destructiveHint, or idempotentHint. Agents cannot determine safety/idempotency of repeated calls or whether state is modified.
Pagination not documented. mbta_departures accepts 'limit' (1-30) but no offset/cursor or total_count guidance. Large result sets risk context window exhaustion.