Loading ToolBench
MCP server for interacting with the San Francisco MUNI (Municipal Transportation) API, providing access to route information, route details, real-time predictions, and cache management
Server has 6 tools with basic schemas and descriptions, but significant gaps limit production readiness. All tools have descriptions (10-80 chars, mostly short), and input schemas are present with type definitions. However, descriptions lack depth, most are single-sentence statements without context on WHEN to use the tool or what happens on failure. Parameter descriptions are minimal (e.g., 'ID of the route' without format guidance). Output schemas are not documented, callers cannot predict response structure. Error handling returns generic error strings without recovery guidance. No tool annotations (readOnlyHint/destructiveHint) despite clear risk distinctions (health_check, list_all_routes, get_route_details, get_predictions are READ_ONLY; clear_cache, toggle_cache are WRITE). Tool composition is reasonable (6 focused tools), but naming could be clearer: 'toggle_cache' is ambiguous, 'enable_cache' and 'disable_cache' would be more explicit.
Clear the cached MUNI API responses
Get real-time arrival/departure predictions for a specific stop on a route
Get detailed information about a specific MUNI route
Check if the MUNI API server is healthy
Get a list of all MUNI routes with detailed information
Enable or disable caching of MUNI API responses
Output schemas not documented. Callers cannot predict response structure (fields, types, pagination). LLMs must infer structure from examples, risking parsing errors.
Descriptions lack context on WHEN to use each tool and what happens on failure. E.g., 'Get a list of all MUNI routes' does not explain when to call this vs get_route_details, or what to do if the API is down.
No tool annotations (readOnlyHint/destructiveHint). clear_cache and toggle_cache are WRITE operations that modify state, but this is not signaled to the LLM via annotations. Agents cannot distinguish safe vs unsafe tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2026-07-28+ | v2 |
Parameter descriptions are minimal and lack format guidance. E.g., route_id: 'ID of the route (e.g., N for N-Judah)' does not specify valid format, length, or case sensitivity. LLMs may pass invalid values.
Error handling returns generic error strings without recovery guidance. E.g., 'Failed to fetch routes: <error>' tells the LLM nothing about whether to retry, call a different tool, or ask the user.