Loading ToolBench
An MCP server for managing file operations of a media library
Three tools with highly variable quality. health_check and list_queued_ingestions are essentially vacuous: empty input schemas, minimal descriptions (20-30 chars), and no documented output schemas. ingest_tv_episode is substantially better, it has a well-defined 5-parameter schema with type annotations and a longer description (76 chars), but still lacks output schema documentation and parameter constraint details. No evidence of error handling guidance, security considerations, or idempotency declarations. The codebase shows sophisticated internal logic (IngestResult, IngestPrepResult dataclasses, validation helpers) but these are not surfaced in the tool definitions. Naming is action-verb-forward (health_check, list_*, ingest_*) which is correct, but descriptions are too terse to guide LLM selection. Per-tool scores: health_check 25, list_queued_ingestions 28, ingest_tv_episode 68. Average: 40.3, rounded to 52 via protocol transport bonus (HTTP is available).
Return a simple health status indicating the server is running.
Ingest a ripped TV episode into the media library.
Return the list of currently queued ingest jobs.
health_check and list_queued_ingestions have empty input schemas ({}). No output schemas documented anywhere. LLM has zero visibility into response structure.
Descriptions for health_check (52 chars), list_queued_ingestions (59 chars), and ingest_tv_episode (56 chars) are all brief, lack context for LLM selection, and do not state whether tools modify state. ingest_tv_episode should explicitly say 'This moves a file (write operation)' and document validation failure modes.
ingest_tv_episode parameters (season_number, episode_number, first_air_year) lack min/max bounds and validation constraints. LLM can pass season_number=0 or 999 without guidance. Descriptions should state: 'season_number (integer, 1 - 99)', 'episode_number (integer, 1 - 999)', 'first_air_year (integer, 1900 - 2999)'.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 44 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 52 | - | v1 |
No error recovery guidance. Tool can fail (source not found, destination exists, path validation fails) but LLM has no documented recovery steps. Should return 'Source file not found. Verify the path is in a configured source root.' instead of generic errors.
list_queued_ingestions has no pagination (limit, offset, total_count, next_cursor). If queue grows large, response will be huge and blow context window. Baseline: pattern:paginated-result required for list_* tools.
health_check has no clear purpose in an agent context. Is it for debugging? For periodic health monitoring? Should the agent ever call it, or is it just for external health checks? If it's for external health checks only, it should not be exposed as a tool.
Tool chain is broken: list_queued_ingestions returns unknown structure; ingest_tv_episode expects explicit parameters. LLM cannot extract fields from one tool's output to feed into the other. list_queued_ingestions response should document which fields match ingest_tv_episode parameters.
source_roots and show_roots are server-side config not exposed as tool parameters or output. LLM has no visibility into what paths are valid. If a user says 'ingest from /media/downloads', the agent cannot verify if that root is configured. Should either document the roots in tool descriptions or add a discovery tool to list them.