Loading ToolBench
An MCP server that reads, writes, and looks up audio file metadata (ID3/Vorbis) for LLM clients over stdio.
Server has 6 well-named tools with clear verb-noun patterns (read_tags, write_tags, scan_directory, lookup_release). Descriptions are present and substantive (100-250 chars), explaining WHAT each tool does and WHEN to use it. Input schemas are visible with typed parameters and descriptions. However, output schemas are NOT documented, LLMs cannot predict return structure. Error handling is implicit (functions return dicts with 'ok' field) but not surfaced in tool descriptions. Missing tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. Dry-run support is good for destructive ops but not highlighted in descriptions.
Remove ALL tags from one file (audio stream untouched). dry_run previews what would be removed.
Look up album/date/track-number candidates from MusicBrainz for a song. Read-only: returns ranked suggestions to fill missing tags. Feed a chosen value into write_tags to apply it. Provide artist (required) and the track title for best matches.
Read normalized tags PLUS every raw frame (incl. custom ones) + stream info from one file.
List audio files in a directory and flag which are missing required fields (default: album).
Update ONLY the provided tag fields on one file; all other tags stay untouched. Writes are saved in place (no undo) — work on copies / keep backups. Set dry_run=True to preview the before/after diff without saving.
Apply the SAME field updates to many files at once (e.g. set one album on a whole folder). Same per-file semantics as write_tags (only provided fields change, in place, no undo). Use dry_run=True to preview. Continues past individual failures.
Output schemas not documented. LLMs cannot predict return structure (fields, types, pagination). read_tags returns 'normalized tags PLUS every raw frame' but structure is opaque. write_tags returns diff but format unknown.
Tool annotations missing. write_tags and clear_tags are destructive (save in place, no undo) but lack destructiveHint. read_tags and lookup_release are read-only but lack readOnlyHint. Agents cannot infer safety without annotations.
Error handling not surfaced in descriptions. tags.py returns error dicts with 'ok' and 'error' fields, but tool descriptions do not explain error cases or recovery paths. LLMs cannot plan error recovery.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Dry-run feature not highlighted in descriptions. write_tags and clear_tags support dry_run=True to preview changes, but descriptions bury this as a parameter rather than emphasizing it as a safety mechanism.
lookup_release requires 'artist' but title is optional. Description does not explain impact: title omission may return poor matches. Should guide LLM to provide both for best results.