Loading ToolBench
MCP server to control Spotify (playback, search, queue, library, playlists) from an AI agent
Spotify MCP has clear tool names and reasonable descriptions, but suffers from significant schema and parameter documentation gaps. All 6 tools use action-dispatch patterns (single parameter controlling behavior) rather than dedicated tool names, forcing LLMs to reason about string values. Parameter descriptions exist but lack constraints (enums, ranges, formats). Output schemas are not documented. Error handling is minimal, handlers return generic text without recovery guidance. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels. Schemas are present and properly typed via Pydantic, but descriptions average ~100 chars (below the 194-char baseline for A+ tools).
Get detailed info about a Spotify item (track, album, artist, or playlist).
Manage saved (liked) tracks. - get_liked: list saved tracks. - save / remove: add or remove track_ids from the library.
Manage playback: get current track, start/resume, pause, skip, previous, seek, set volume. - get: info about the current track. - start: play spotify_uri, or resume if omitted. - pause / skip / previous. - seek: jump to position_ms. - volume: set volume_percent (0-100).
Manage playlists. - list: the current user's playlists. - create: make a new playlist (name required). - add_tracks / remove_tracks: modify a playlist's items (playlist_id + track_ids required). - delete: remove a playlist from your library (playlist_id required). Spotify has no hard delete; this unfollows it, which removes owned playlists from your view.
Manage the playback queue - get the queue or add a track.
Search for tracks, albums, artists, or playlists on Spotify.
Action-dispatch anti-pattern: All 6 tools use a single 'action' parameter with string values (e.g., 'get', 'start', 'pause') instead of separate verb-prefixed tools. This forces LLMs to reason about string matching and violates the verb_noun naming convention. Example: SpotifyPlayback should split into get_current_track, start_playback, pause_playback, skip_track, etc.
Missing enum constraints on action parameters. 'action' fields accept free-form strings with no enum declaration. LLMs cannot validate options and may hallucinate invalid values like 'resume' instead of 'start'. All action parameters should declare enums: {"enum": ["get", "start", "pause", ...]}.
No output schema documentation. Tool descriptions do not specify what fields are returned or their types. LLMs cannot plan downstream calls or extract data reliably. Example: SpotifySearch returns results but the response structure (track objects, fields, pagination) is undocumented.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | <=2025-11-25 | v2 |
Missing tool annotations. Tools with WRITE and DESTRUCTIVE risk levels (SpotifyPlayback, SpotifyQueue, SpotifyLibrary, SpotifyPlaylist) lack readOnlyHint/destructiveHint annotations. Clients cannot distinguish safe read-only tools from dangerous ones.
Minimal error handling and recovery guidance. Handlers return generic text ('Unknown action', 'track_id is required') without actionable next steps. No categorization of errors as retryable vs. user-fixable. Example: handle_playback returns 'Unknown playback action: {action}' instead of 'Unknown action. Valid options: get, start, pause, skip, previous, seek, volume.'