Loading ToolBench
Four tools with complete schemas and reasonable descriptions. Naming follows verb_noun pattern (get_, search_, top_). Descriptions are 150-250 chars, within baseline range (p10=34, p90=392). All parameters have type definitions and descriptions. However, output schemas are not documented, LLMs cannot predict response structure. Error handling returns structured objects but lacks recovery guidance. The _apiKey parameter pattern is unconventional (underscore prefix) and exposes credential handling in tool interface, violating secret-injection pattern. No tool annotations (readOnlyHint, etc.) despite all being read-only operations.
Get live Twitch streams, optionally filtered by game name and/or language. Returns streamer name, title, current viewer count, and thumbnail for each live stream. Omit `game` for the overall top live streams across Twitch. Example: get_streams({ game: "Just Chatting", first: 20, language: "en" })
Look up a Twitch user/channel by login (username). Returns profile info: id, display name, description, view count, broadcaster type, account creation date, and profile image. Example: get_user({ login: "ninja" })
Search Twitch channels/streamers by name or keyword. Returns matching channels with display name, the game they stream, whether they are live now, title, and thumbnail. Example: search_channels({ query: "ninja", live_only: true })
Get the top games currently being streamed on Twitch, ranked by number of viewers. Returns game id, name, and box art. Example: top_games({ first: 20 })
Output schemas not documented. LLMs cannot predict response structure (fields, types, nesting). Responses are inferred from code (e.g., topGames returns {games: [{id, name, box_art}]}) but not declared in tool definitions.
_apiKey parameter exposes credential handling in tool interface. Credentials should be injected server-side via environment variables or vault, not passed as tool parameters. Agent traces log all parameters, secrets leak into logs and prompt history.
No tool annotations. All four tools are read-only (GET operations), but lack readOnlyHint annotation. Agents cannot distinguish safe tools from destructive ones without explicit hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Error responses lack recovery guidance. When credentials fail or API returns non-2xx, responses return {error, message} but do not suggest next steps (e.g., 'Check your CLIENT_ID:CLIENT_SECRET format' or 'Retry with exponential backoff').
No pagination support. get_streams and search_channels accept 'first' parameter but do not return cursor, total_count, or has_more. Large result sets cannot be paginated, risking context window exhaustion.