Loading ToolBench
MCP server for audio transcription with word-level timestamps, phoneme-level detail, speaker diarization, and real-time performance metrics
Critical failures across all dimensions. The single tool 'execute_skill_action' has a minimal description (67 chars, below baseline 194), but the core issue is that the MCP server implementation is non-functional. The mcp_server.py file contains only a print statement that outputs JSON, it does not implement the MCP protocol at all. No tool registration, no request/response handling, no schema validation. The tool definition exists only in documentation; the actual server cannot receive or process tool calls. Input schema is visible (input_audio_file, diarization_enabled with types), but the server has no mechanism to invoke it. This is a stub implementation, not a working MCP server.
Execute audio transcription with word-level alignment and phoneme-level detail
MCP server does not implement the protocol. mcp_server.py contains only a single print() statement outputting static JSON. No request/response loop, no tool invocation handler, no message parsing. The server cannot receive or process tool calls from any MCP client.
Tool description is too short (67 chars, baseline 194). 'Execute audio transcription with word-level alignment and phoneme-level detail' lacks context on WHEN to use it, what it returns, or prerequisites. Does not meet the 10 - 1024 char guideline or LLM-optimization baseline.
Input parameters lack descriptions. 'input_audio_file' and 'diarization_enabled' have type info but no descriptions explaining what they control, expected format, or constraints. LLMs cannot infer parameter meaning from names alone.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | 2026-07-28+ | v2 |
Output schema is not documented. The client.py mock returns a dict with fields like 'transcription_job_id', 'word_error_rate_wer_pct', 'speakers_diarized_count', etc., but the tool definition does not declare this structure. LLMs cannot plan downstream calls without knowing what fields to expect.
No error handling or recovery guidance. The tool definition does not specify what errors can occur (invalid audio file, unsupported format, service timeout) or how the LLM should respond. No actionable error messages.