Loading ToolBench
Extract public Spotify data — tracks, albums, artists, playlists, podcasts, lyrics, charts, related artists, recommendations, Canvas videos, cover colors, credits, concerts, and public user profiles — with no official API key.
SpotifyScraper demonstrates solid definition quality with consistent naming, present descriptions, and complete JSON Schema input definitions across all 27 tools. All tools follow verb_noun conventions (get_*, list_*, search_*). Tool descriptions are generally actionable (60-180 chars, within the 10-1024 baseline). Input schemas include type definitions and descriptions for all parameters. However, output schemas are not explicitly documented in the source, LLMs cannot verify what fields to expect from responses. Descriptions lack dependency hints and error guidance. No parameter constraints (enums, patterns, ranges) are declared, leaving validation to the runtime. Error responses are not documented. These gaps prevent this from reaching 80+.
Fetch the authenticated user's account metadata (needs SPOTIFY_SP_DC).
Fetch an album (with its tracks) by URL, URI, or ID.
Fetch many albums at once; one ordered result per input, failures captured per item.
Fetch an artist by URL, URI, or ID.
Fetch an artist's upcoming concerts/events.
Fetch many artists at once; one ordered result per input, failures captured per item.
Fetch a track's Canvas looping video (needs SPOTIFY_SP_DC).
Output schemas not documented. Tool descriptions do not specify what fields LLMs can expect from responses. This prevents agents from planning downstream tool calls or extracting specific fields reliably.
No input constraints declared (enums, patterns, min/max ranges). For example, search() accepts a 'types' array with free-form strings; valid values (track, album, artist, playlist, show, episode) are not declared as an enum. This leaves validation to runtime and invites LLM hallucination of invalid values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | A | 84 | 2026-07-28+ | v2 |
Fetch an editorial chart (e.g. 'top-50-global') as a playlist.
Extract a cover image's theming colors (image URL/uri, or any entity URL/URI/ID).
Fetch a track's credits — performers, writers, and producers (needs SPOTIFY_SP_DC).
Fetch an artist's full discography (albums, singles, compilations).
Fetch a podcast episode by URL, URI, or ID.
Fetch many episodes at once; one ordered result per input, failures captured per item.
Fetch a track's lyrics (needs SPOTIFY_SP_DC).
Fetch a playlist by URL, URI, or ID (up to ``max_tracks`` tracks).
Fetch many playlists at once (up to ``max_tracks`` each); failures captured per item.
Fetch artists related to an artist ('fans also like').
Fetch a podcast show (with episodes) by URL, URI, or ID.
Fetch many shows at once (up to ``max_episodes`` each); failures captured per item.
Recommend albums similar to a track.
Fetch a track by URL, URI, or 22-character ID.
Fetch a track with its cover colors and Canvas in one call (Canvas needs SPOTIFY_SP_DC).
Fetch many tracks at once; one ordered result per input, failures captured per item.
Fetch an episode's transcript (needs SPOTIFY_SP_DC).
Fetch a public user profile (needs SPOTIFY_SP_DC for private profiles or current user).
List the built-in editorial charts (key, name, backing playlist id).
Search across tracks, albums, artists, playlists, shows, and episodes.
Authenticated tools (get_canvas, get_credits, get_lyrics, get_transcript, get_account, get_user) require SPOTIFY_SP_DC cookie but descriptions do not explicitly state 'requires authentication' or document the error behavior when the credential is missing. LLMs cannot prepare for authentication failures.
Batch tools (get_tracks, get_albums, get_artists, get_episodes, get_playlists, get_shows) claim to capture per-item failures but do not document the response structure for partial failures. How are errors returned? As null? As an error object? This ambiguity prevents agents from handling failures correctly.
Paginated results (list_charts, search, get_playlist, get_show, get_discography) accept limit/max parameters but do not document pagination mechanism (offset, cursor, next_token). Can LLMs fetch the next page? How?
Many parameter descriptions are minimal (e.g., 'Fetch a cover image's theming colors (image URL/uri, or any entity URL/URI/ID)' for get_colors). They lack guidance on what input format is expected, when to use the tool instead of alternatives, and what the next action might be.
Error handling and recovery guidance missing. No documented error responses or actionable recovery instructions (e.g., 'If track not found, call search() with track name').