Loading ToolBench
Washington Metro real-time + static rail/bus data. Free key required.
Critical deficiencies across all dimensions. Tool descriptions are extremely terse (5-25 chars, well below 34-char p10 baseline). Parameter descriptions are missing entirely for most tools, only 3 of 10 tools have ANY parameter descriptions, and those are minimal ('Optional line code to filter stations'). No input schemas visible for 5 tools (rail_lines, rail_incidents, bus_routes, bus_incidents lack properties). Output schemas completely undocumented. No error handling guidance. Tool names lack action verbs (rail_lines, rail_stations, rail_predictions are noun-heavy; bus_routes, bus_stops are ambiguous). API key injection via _apiKey parameter violates secret-injection pattern. No pagination support despite APIs likely returning large result sets. No per-tool output documentation.
Bus incidents.
Bus arrival predictions.
Route details.
List bus routes.
Nearby bus stops.
Current rail incidents.
List rail lines.
API key exposed as tool parameter (_apiKey). Violates secret-injection pattern, credentials in parameters leak into logs and prompt history. Must use server-side environment variable injection (PLATFORM_WMATA_KEY).
Tool descriptions critically short (5-25 chars, baseline p10=34). 'List rail lines', 'List stations', 'Bus incidents' provide no context for LLM selection. Must explain WHAT, WHEN, and WHAT IT RETURNS (50-200 chars target).
Parameter descriptions missing or minimal. Only 3 of 10 tools have param descriptions; most are 1-2 words. 'Optional line code to filter stations' is bare minimum. Must document format, constraints, and dependencies for every parameter.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 35 | 2026-07-28+ | v2 |
Next-train predictions.
Station info.
List stations.
No output schemas documented. LLMs cannot plan downstream calls or extract data without knowing response structure. Must document return types, field names, and pagination info for each tool.
No pagination support. WMATA APIs likely return large result sets (all rail lines, all bus routes, all stops). Without limit/offset/cursor, responses blow context window. Must add page_size, offset, and total_count to list tools.