Loading ToolBench
Strong foundation with 42 well-named tools covering Spotify's API comprehensively. All tools have descriptions (10-200 chars, baseline 194) and explicit JSON Schema input parameters with types. Naming follows verb_noun convention (search, play, pause, get, list). However, output schemas are not documented, responses are raw JSON or text without field descriptions. Error handling is minimal (generic error strings, no recovery guidance). Parameters lack constraints (enums, ranges, format specs) that would prevent LLM hallucination. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear read/write distinction. Descriptions are functional but terse (avg ~80 chars vs 194 baseline), missing WHEN to use, prerequisites, and dependency hints.
Get a single album's metadata (includes its track list).
List the tracks on an album (paginated).
List an artist's albums.
Get a single artist's metadata.
List Spotify Connect devices. is_restricted=true devices (e.g. speakers) only accept transfer_playback, not direct play/queue/skip.
Get a single episode's metadata.
List the user's saved albums.
Output schemas not documented. Tools return raw JSON or text without field descriptions. LLMs cannot plan downstream calls or extract required data reliably.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Tools are marked READ_ONLY or WRITE in metadata but not in MCP schema. Agents cannot distinguish safe from destructive operations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
Remove one or more items (tracks, albums, shows, episodes, audiobooks) from the user's library.
Save one or more items (tracks, albums, shows, episodes, audiobooks) to the user's library.
List the user's saved tracks.
Get the current user's Spotify profile (id, country/market, product).
Skip to next track (fails on restricted devices).
Get the currently-playing item (lighter than playback_state).
Pause playback.
Start/resume playback. Provide context_uri (album/playlist/artist) OR uris (tracks). Both empty = resume.
Get current playback state: active device (with is_restricted), track, progress.
Add tracks to a playlist.
Update a playlist's name, description, or public flag.
Create a playlist for the current user.
Get a playlist's metadata.
Get items of a playlist.
Remove tracks from a playlist.
List the current user's playlists.
Skip to previous track.
Add a track to the playback queue. Requires a non-restricted active device.
Get the upcoming queue.
Get recently-played history.
Get recommendations from explicit seeds (artist names/ids/URIs, track ids/URIs, and/or genre names).
Get recommendations based on the user's top artists and tracks.
Set repeat mode: track (repeat one), context (repeat album/playlist), or off.
List the user's saved audiobooks.
List the user's saved episodes.
List the user's saved shows (podcasts).
Search Spotify. limit is capped at 10 (Feb-2026 API). Use offset to paginate.
Search tracks and return a slim list (id, uri, name, artists). limit capped at 10.
Seek to a position in the current track.
Set playback volume (0-100).
List a show's episodes (paginated, newest first).
Get a single show's (podcast's) metadata.
Toggle shuffle on/off for playback.
Get a single track's metadata.
Move playback to a device by id. The correct way to drive restricted devices (speakers). Device must be online/awake.
Parameter descriptions are terse (avg ~80 chars vs 194 baseline). Missing WHEN to use, prerequisites, and dependency hints. E.g., 'search' lacks guidance on field filters or when to use search vs search_tracks.
No input validation or constraint documentation. Parameters lack enums, ranges, or format specs. E.g., 'limit' in search is capped at 10 but no enum/max constraint in schema; 'percent' in set_volume lacks 0-100 range constraint.
Error handling is minimal. Errors return generic strings with no recovery guidance. E.g., 'Playback transferred.' vs 'Device offline, try devices_list() to find an active device.'