Loading ToolBench
MCP server for fetching YouTube video transcripts, extracting video IDs, and summarizing videos with Claude CLI integration
The server has 8 tools with basic descriptions and parameter definitions, but significant quality gaps reduce the score substantially. Tool naming is mostly clear (verb-noun pattern), but descriptions are inconsistent in quality and depth. Parameter schemas are present but lack comprehensive validation constraints. Most critically, several tools mix multiple responsibilities (e.g., get_transcript_for_summary does fetch + template generation, publish_summary does classification + file management + git operations), and error handling guidance is absent. Output schemas are not documented. The server exhibits a C/D-tier implementation typical of early-stage community projects.
Create a prompt for analyzing YouTube video transcript.
Extract YouTube video ID from URL.
Get transcript from YouTube video URL.
Get transcript and prompt template for summarizing a YouTube video. Use this tool to fetch a video transcript along with the summarization prompt. After receiving the response, generate a comprehensive summary following the prompt, then call publish_summary with the result.
Get transcript metadata as a resource using video ID.
Get transcript as a resource using video ID.
Publish a video summary to GitHub. This tool takes a generated summary, classifies it, generates a filename, saves it to the appropriate category folder, and pushes to GitHub.
Multiple tools bundle unrelated responsibilities, violating single-responsibility principle. Example: get_transcript_for_summary fetches transcript AND generates prompt template AND returns instructions for downstream summarization, three distinct concerns. Example: publish_summary classifies content, generates filename, writes to directory, and pushes to GitHub, four distinct operations.
Output schemas are not documented in tool descriptions or visible in code. LLMs cannot predict what fields to expect from responses (e.g., does get_transcript return a flat text string, a JSON object with metadata, structured segments with timestamps?). This violates the documented output schema requirement.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 55 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Create a prompt for summarizing YouTube video transcript.
Error handling lacks actionable guidance. No tool description explains what the LLM should do if the tool fails (e.g., 'If video ID extraction fails, try passing a full YouTube URL instead of a short link' or 'If transcript fetch fails with 403, the video may have disabled captions'). Bare error responses force LLMs to guess next steps.
Parameter descriptions are minimal. Example: 'languages' param in get_transcript is described as 'Comma-separated list of preferred languages (default: "en")' but does not explain which language codes are valid (ISO 639-1? YouTube's custom codes?) or what happens if a language is unavailable (fallback to English? Error? Return available languages?).
Tool naming ambiguity: three separate tools (get_transcript, get_transcript_resource, get_transcript_for_summary) perform similar operations with unclear distinctions. LLMs will struggle to pick the right one. 'get_transcript_resource' and 'get_transcript_for_summary' names do not convey their actual purpose clearly enough to disambiguate at a glance.
Destructive operation (publish_summary) lacks confirmation or dry-run capability. An LLM could accidentally publish a malformed summary to GitHub without an intermediate review step. No mention of permission checks or git auth validation in descriptions.
Parameter validation constraints are implicit or missing. Example: publish_summary accepts 'video_url' but does not validate format or check whether it is a real YouTube URL. Does the tool accept any string? The description does not constrain it.
Prompts (analyze_transcript, summarize_transcript) operate on prompts mode but output is not clearly documented. Are they returning a prompt string to pass to Claude? Or are they returning analysis/summary directly? Ambiguity around prompt tools reduces clarity.