Loading ToolBench
Search YouTube Music, build song radios, and create private playlists.
Strong naming conventions (verb_noun pattern), comprehensive parameter descriptions, and well-structured schemas across all 6 tools. Tool annotations present for read-only vs write operations. However, output schemas are defined via TypedDict but not exposed in tool registration metadata, limiting LLM visibility into response structure. Parameter validation is robust with custom validators (_query, _limit, _playlist_title, _playlist_id). Error messages are actionable. Main gaps: output schemas not documented in tool definitions themselves, and some parameter descriptions could be more explicit about constraints (e.g., playlist_id format details).
Create one private playlist from a fresh two-to-ten-seed radio mix.
Create a private playlist from a fresh song-radio queue in the connected account.
Round-robin two to ten song radios into one deduplicated result.
Get YouTube Music's ordered radio recommendations around the first song match.
Resume one exact saved track plan; do not invoke concurrently.
Find songs with IDs, artists, album titles, durations, and links.
Output schemas defined in TypedDict but not exposed in MCP tool registration. LLMs cannot see what fields to expect from responses, forcing them to infer structure or make follow-up calls.
resume_private_playlist description lacks clarity on what 'resume' means and why concurrent invocation is forbidden. Should explain: resumes adding tracks from a saved plan, and concurrent calls risk duplicate additions.
playlist_id parameter description mentions '10-128 character alphanumeric string' but validation logic (normalize_playlist_id) may accept other formats. Constraint description should match actual validation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
No pagination support documented for search_songs or radio tools. If results exceed limit, no guidance on how to fetch additional results or whether more exist.
Error handling returns ToolError with messages, but no categorization (retryable vs user-fixable vs fatal). LLM cannot determine if it should retry, ask user, or abandon.