Loading ToolBench
Skills for building and testing voice AI agents with Vapi, including assistants, tools, calls, campaigns, squads, phone numbers, webhooks, structured outputs, prompts, and simulations.
This MCP server has 11 tools defined in a Python build script (scripts/build-codex-plugin.py). The server is evaluated as STDIO-only, meaning it cannot be remotely accessed. Tool definitions are INFERRED from a configuration dictionary (SKILL_INTERFACES) rather than explicitly registered with JSON Schema definitions. No input schemas are visible in the source code provided. Descriptions exist but are extremely brief (15-40 characters), far below the 50-200 character baseline for LLM-optimized descriptions. No parameter definitions, types, or constraints are documented. No output schemas are specified. Error handling is not evident. Security considerations around API key management are mentioned in tool names but not addressed in schema or validation. The source shows a Codex plugin build script, not an MCP server implementation with proper tool registration.
Design Vapi voice assistant configurations.
Prepare one-off Vapi phone or web calls.
Plan and manage persistent outbound campaigns.
Configure Vapi numbers and provider imports.
Design multi-assistant Vapi handoff workflows.
Design reusable Vapi post-call extraction.
Design tools for Vapi voice assistants.
No input schemas visible in source code. All 11 tools inferred from configuration dict only, not explicitly registered with JSON Schema definitions. Per scoring rule: 'If you cannot see the actual tool definition in the source (only inferred): cap that tool's overall at 50.' With zero visible schemas, all schema scores are 0.
Descriptions are extremely brief (15 - 40 characters), far below the 50 - 200 character LLM-optimized baseline. Examples: 'Configure a private Vapi API key safely.' (43 chars), 'Design Vapi voice assistant configurations.' (42 chars). Brevity prevents LLMs from understanding WHEN to call these tools vs. similar ones, and what to expect in return.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 37 | 2026-07-28+ | v2 |
Configure a private Vapi API key safely.
Configure Vapi server URLs and call events.
Design Vapi assistant simulation suites.
Write and audit production voice-agent prompts.
No parameter definitions visible. Tools accept user input (evidenced by names like 'create-assistant', 'create-call') but no parameters are documented with types, descriptions, enums, or constraints. This forces LLMs to guess what inputs are required, their format, and valid values.
No output schemas documented. LLMs cannot know what fields to expect from tools like 'create-assistant' or 'create-call', preventing composition of downstream tools that might need IDs, status, or configuration details.
No visible error handling or recovery guidance. If a tool call fails (e.g., invalid API key, missing required field), LLMs have no guidance on what to do next: retry, ask the user, or abort.
'setup-api-key' tool invites API key exposure. Source shows this is a configuration tool, but if implemented as a writable parameter, it violates the secret-injection pattern. API keys must never appear in tool parameters; they should be injected server-side via environment variables or secure vaults.
Tool names contain generic words ('create') without distinguishing specifics. 'create-tool' is ambiguous, does it create a Vapi tool (for voice assistants) or a generic tool? More specific naming like 'create-vapi-tool' or 'create-voice-tool' would clarify intent.
No documentation of tool composition or chaining. If 'create-assistant' returns an assistant_id, downstream tools like 'create-call' should accept that ID. This relationship is not documented, forcing LLMs to infer or guess.