Loading ToolBench
OpenF1 MCP demonstrates solid fundamentals: all 4 tools have clear verb-noun naming (get_*), complete input schemas with type definitions, and descriptions (avg 180 chars). However, output schemas are undocumented, LLMs cannot predict response structure for planning. Parameter descriptions lack constraints (enums, ranges, formats). Error handling is minimal: no recovery guidance, no categorization of retryable vs fatal errors. The get_laps tool hardcodes a 200-item limit without documenting it in the description. Missing tool annotations (readOnlyHint, idempotentHint) despite all tools being read-only.
List F1 drivers entered in a session (live race data): number, full name, 3-letter acronym, team, team colour, country, headshot. session_key="latest" for the most recent / ongoing session.
Lap times / telemetry for an F1 session: per-lap duration, sector times, speed-trap, pit-out flag. Live race data. session_key="latest" for the most recent / ongoing session. Optionally filter by driver_number or lap_number.
List Formula 1 meetings (Grand Prix weekends). Live race data source. Filter by year and/or country. Returns meeting_key, official name, country, circuit, location, date.
List F1 sessions (a Race, Qualifying, or Practice within a Grand Prix weekend). Use to find the session_key needed for drivers/laps. Filter by year, country, session_name ("Race","Qualifying","Practice 1"), or meeting_key.
Output schemas not documented. LLMs cannot predict response structure (fields, types, nesting) for downstream planning or data extraction. Responses are shaped inside callTool but never exposed to the agent.
Parameter descriptions lack formal constraints. 'year' accepts any number; should specify range (e.g. 1950 - 2099). 'country_name' is free-form string; should list valid countries or reference enum. 'session_name' example lists only 3 types but doesn't declare them as the only valid options.
No error recovery guidance. f1Get returns {error, message} on HTTP failure, but callTool does not catch or transform this into actionable guidance. LLM receives raw error with no hint on retry strategy, alternative tools, or user-fixable causes.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
get_laps silently caps results at 200 items without documenting this limit in the tool description. LLM may assume all laps are returned and make incorrect inferences. Pagination not supported.
Tool annotations missing. All 4 tools are read-only (no state mutation), but readOnlyHint is not set. Agents cannot distinguish safe-to-retry tools from destructive ones without explicit hints.