Loading ToolBench
Exposes YouTube channel operations as MCP tools: identity/listing, publishing (upload, metadata, thumbnails), pseudo A/B thumbnail testing, and analytics.
Server has 9 tools with mostly complete schemas and descriptions. Naming follows verb_noun convention (youtube_whoami, youtube_upload, etc.). Descriptions are present and substantive (avg ~150 chars), explaining WHAT each tool does and WHEN to use it. However, several critical gaps: (1) Parameter descriptions lack format constraints and validation rules, e.g., 'max_results' has no min/max bounds; 'privacy' accepts free-form string instead of enum; 'publish_at' lacks ISO 8601 format validation. (2) Output schemas are not formally documented, tools return JSON strings but no schema is declared for downstream tool chaining. (3) Error handling is minimal, no recovery guidance or actionable error messages. (4) A/B test tools (youtube_ab_start, youtube_ab_rotate, youtube_ab_report) have honest caveats in descriptions but lack error classification and retry guidance. (5) No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels (READ_ONLY vs WRITE). Composition is sound, each tool has one responsibility and output includes IDs for chaining (video_id returned by upload, used by update_metadata). Security: credentials handled server-side via auth module, no secrets in params.
Compare A/B test variants by views / watch-time per window. Returns a summary table of each variant's performance across windows.
Advance a pseudo-A/B thumbnail test one step: close the current window, set the next variant live, and open the next window. Call this when a window ends (or schedule via launchd/cron).
Begin a pseudo thumbnail A/B test. Records a ledger of the variant image paths, sets the FIRST variant live now, and opens window 0. Call youtube_ab_rotate when a window ends (or schedule ab_rotate.py via launchd/cron) to advance to the next variant. CAVEAT: YouTube's native thumbnail A/B ("Test & Compare") has no public API, and impressions/CTR are not exposed by the Analytics API. This measures views gained per equal-length window as a CTR proxy, so use windows >= 24h and keep all other variables constant. Not a statistically rigorous split test.
Fetch analytics for a video: views, watch time, avg view %, subs, etc.
List the channel's most recent uploads with their video ids, titles, privacy status and publish time. Use the ids with the edit/analytics tools.
No enum constraints on string parameters. 'privacy' accepts free-form string (should be enum: private|unlisted|public); 'category_id' accepts string without validation. LLMs will hallucinate invalid values.
Numeric parameters lack bounds. 'max_results' has no min/max declared (description says 'default 10, max 50' but schema has no constraint); 'hours_per_variant' unbounded. LLMs may pass absurd values.
Output schemas not formally documented. Tools return JSON strings but no schema is declared for what fields to expect. Downstream tool chaining requires LLM to parse unstructured output.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Set a custom thumbnail (JPG/PNG, <2MB, 1280x720 recommended). Channel must be verified for custom thumbnails to be accepted.
Edit an existing video's metadata. Only the fields you pass change; the rest are preserved (the API requires sending the full snippet, so this reads the current values first and merges).
Upload a local video file (resumable).
Return the authorized channel's id, title, subscriber/video counts, and whether custom thumbnails are allowed (channel must be phone-verified).
No tool annotations despite clear risk levels. Tools marked WRITE (youtube_upload, youtube_update_metadata, youtube_set_thumbnail, youtube_ab_start, youtube_ab_rotate) lack destructiveHint annotation. READ_ONLY tools lack readOnlyHint. Agents cannot distinguish safe from risky operations.
Minimal error handling and recovery guidance. No error classification (retryable vs user-fixable vs fatal). A/B test tools include honest caveats but no actionable error messages or retry logic.