Loading ToolBench
Hosted MCP server for music service integrations, specifically Apple Music
Scoring was not performed
Parameter descriptions are sparse or missing. 'limit' and 'offset' appear in list_playlists, get_library_songs with no descriptions. 'types' in search_catalog has minimal context. Zod definitions exist but LLMs rely on description text, not schema introspection.
Output schemas are completely undocumented. All tools return JSON via generic 'JSON.stringify(result, null, 2)' but the response structure is never defined. LLMs cannot plan downstream calls without knowing what fields to expect (e.g., does search_catalog return 'id' or 'trackId'?).
Error handling returns bare error messages without recovery guidance. All 8 tools catch errors with: `Error: ${err.message}` and return isError: true. No indication of whether error is retryable, user-fixable, or fatal. No suggestions for next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 21 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 58 | - | v1 |
No input validation or constraint documentation. search_catalog accepts 'types' as comma-separated string but has no validation, pattern, or enum. 'limit' has min/max in Zod (1-25) but not explained in description. LLMs cannot infer valid values from source code alone.
Irreversible operations (create_playlist, add_tracks) include warnings in descriptions but no confirmation mechanism. Description says 'Please confirm with the user before calling this tool' but no dry-run or confirmation_token parameter exists. Warning alone does not prevent accidental execution.
No pagination limit enforcement. list_playlists and get_library_songs accept 'limit' and 'offset' without documented max limits or defaults. get_playlist_tracks claims 'Handles pagination automatically' but no pagination parameters are exposed, behavior is opaque to the caller.