Loading ToolBench
A custom MCP server that performs million-token multimodal needle retrieval from video streams, locating specific temporal events with high confidence in interleaved audio/video content
This server presents severe structural problems. Only one tool is defined (execute_skill_action), and while it has a name and description, the implementation is not an actual MCP server, it's a stub that prints JSON to stdout and does not implement the Model Context Protocol specification. The tool description is present but lacks critical details about parameters, output schema, and error handling. The input schema is provided in the evaluation metadata but is not visible in the actual source code (mcp_server.py contains no schema definitions, tool registration, or parameter validation). The tool cannot be traced to an explicit MCP registration in the codebase. The example usage shows a client class that returns mock data, not a real implementation. Most critically, there is no evidence of: (1) proper MCP server initialization or tool registration, (2) input validation or error handling, (3) documented output schemas, (4) parameter descriptions beyond the evaluation summary, (5) security considerations, or (6) any connection between the 'mcp_server.py' stub and the 'client.py' mock implementation.
Execute a million-token multimodal needle retrieval skill action to locate specific temporal events in video streams
Not a functional MCP server. mcp_server.py contains only a stub that prints a JSON status message and does not implement the MCP server protocol, register tools, or handle tool calls.
Input schema not visible in source code. The schema provided in the evaluation (video_stream_url, temporal_query) does not appear in mcp_server.py. Tool definitions must be explicitly registered with full JSON Schema, inferring tool existence from metadata alone caps the score at 50.
No output schema documentation. The mock client returns a complex object (retrieval_session_id, context_tokens_scanned, needle_timestamp_offset_sec, modality_source, retrieval_confidence_score_pct, evidence_frame_permalink) but the actual MCP server implementation does not document this structure. LLMs cannot plan downstream tool calls without knowing what fields to expect.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 28 | 2026-07-28+ | v2 |
Parameter descriptions missing from source code. While the evaluation summary lists descriptions for video_stream_url and temporal_query, these do not appear in the actual mcp_server.py implementation. Parameter descriptions are required so LLMs understand what values to pass.
No error handling or validation. The stub server does not validate inputs, define error responses, or provide recovery guidance. A malformed URL or empty query produces no documented error path.
Mock implementation disconnected from server. client.py and example_usage.py use a mock MillionTokenMultimodalNeedleRetrievalClient that returns hardcoded data. The actual mcp_server.py does not instantiate or call this client, so there is no real request/response pipeline.
No tool registration or handler. The MCP specification requires explicit tool registration with a handler that processes incoming tool calls. mcp_server.py does not implement this, it only prints a startup message.
Defaults may cause unintended behavior. Both input parameters have defaults (space_mission_archive_4k.mp4 and a specific thruster query), but these are baked into the client class, not the server tool definition. When an LLM omits a parameter, it will receive hardcoded example data instead of an error or a prompt to provide real input.
Tool name does not follow verb_noun convention. 'execute_skill_action' is a generic wrapper that mixes 'execute' (do something) with 'skill_action' (what is it doing?). A clearer name like 'retrieve_video_event_timestamp' or 'locate_video_event' would better signal intent.