Loading ToolBench
The Sora MCP server has 5 tools with reasonable naming (verb-first convention) and present schemas. However, significant gaps exist: parameter descriptions are minimal or missing detail about formats/constraints, output schemas are not documented, error handling provides no recovery guidance, and no tool annotations (readOnlyHint/destructiveHint/idempotentHint) are declared. Tool names are clear but descriptions lack guidance on when to use each tool vs. alternatives. The 'seconds' and 'size' parameters use string types instead of proper numeric/enum constraints, and no validation error messages guide LLM self-correction. The server uses Zod for parsing but does not expose structured error responses. Average description length (~100 chars) is below the baseline (194 chars), and descriptions lack dependency hints or prerequisite guidance. The structuredContent response pattern is present but output field documentation is absent.
Generate a video using OpenAI Sora 2 API
Get the download instructions and authenticated URL for a completed video
Check the status and details of a video generation job
List all video generation jobs with pagination support
Create a remix of an existing video using OpenAI Sora 2 API
Parameter format constraints missing: 'seconds' and 'size' are declared as strings without validation rules, format patterns, or enum constraints. LLMs will hallucinate invalid values like 'seconds: 999' or 'size: foo'. Should use numeric type for seconds with min/max bounds, and enum or pattern for size.
No output schemas documented. Tools return structuredContent but LLM has no specification of what fields to expect (video_id, status, url, error codes, etc.). This forces LLMs to guess field names and risks failed downstream chaining.
Error handling provides no recovery guidance. Catch blocks return bare error text ('Error creating video: ...'). LLM cannot determine if error is retryable, user-fixable, or fatal, or what action to take next.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 63 | 2025-06-18+ | v2 |
| 2026-03-09 | F | 25 | - | v1 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). create-video and remix-video have side effects and should be marked with destructiveHint=true. get-video-status, list-videos, download-video should have readOnlyHint=true. This metadata helps agents plan retry and composition strategies.
Descriptions are brief (60-65 chars avg) and lack dependency context. E.g., remix-video and download-video require a video_id, but descriptions do not mention that get-video-status or list-videos must be called first to obtain one. Missing prerequisite guidance.
'input_reference' parameter in create-video is optional and vaguely described. No specification of valid file types, paths, size limits, or whether it accepts URLs vs. local file paths. Current code comment says 'for now, pass as string' indicating incomplete implementation.
list-videos pagination relies on cursor-based 'after' parameter but total count is not returned in response. Without knowing total items, agent cannot reason about remaining results or determine stopping conditions.
No confirmation/dry-run pattern for destructive operations. create-video and remix-video initiate video generation (WRITE operations with latency and cost). Agents should be able to preview parameters before committing.