Loading ToolBench
MCP server that integrates with Codex CLI for executing tasks and managing sessions with comprehensive error handling, security policies, and progress tracking
Codex MCP server has a mixed definition quality profile. The primary tool (codex) includes comprehensive input validation schema with 12 parameters, sandbox policies, and session management, strong on parameter enumeration and constraints. However, descriptions lack depth and specificity for agent decision-making. The codex tool description (57 chars) is borderline acceptable but does not explain WHEN to use it or its prerequisites. Session management tools (list_sessions, get_session, tail_session, cancel_session) have minimal descriptions (22-48 chars) that lack context for LLM reasoning. Parameter descriptions are present but generic. The stats tool is undocumented. Output schemas are not formally documented in the code examined. No recovery guidance in error handling. Naming follows verb_noun convention well. Overall, the server provides functional tool definitions with good parameter constraints but lacks the LLM-optimized descriptions and output documentation needed for confident agent selection and chaining.
Cancel a running Codex session by SESSION_ID
Execute a task using Codex CLI with comprehensive input validation, sandbox policies, session management, and change tracking
Retrieve details for a specific Codex session by SESSION_ID
List all tracked Codex sessions with their current state and metadata
Get server statistics including uptime and metrics
Stream or tail the output of a running Codex session
stats tool has no description. LLMs cannot determine when or why to call this tool without a description explaining its purpose and output.
Session management tool descriptions are extremely sparse (22-48 characters), below the 34-char p10 baseline for production tools. Descriptions like 'List all tracked Codex sessions with their current state and metadata' (58 chars) lack actionable guidance on WHEN to call the tool relative to alternatives or what to do with the returned data.
Output schemas are not documented in the source code examined. The test file shows assertions on StructuredContent structure (e.g., {sessions: []}, {found: bool}), but no formal output schema is declared in tool definitions. LLMs need documented return schemas to plan downstream operations.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2025-06-18+ | v2 |
| 2026-03-09 | F | 40 | - | v1 |
The codex tool description (57 chars) does not explain WHEN to use it instead of running a local CLI tool, what prerequisites are needed (Git repo, Codex CLI installed), or what the return value format is. The description reads as a feature list, not LLM guidance.
Error handling lacks recovery guidance. No indication of which errors are retryable (e.g., timeout, transient Codex CLI failure) vs. fatal (e.g., invalid sandbox policy). Agents cannot self-correct without actionable error messages.
The codex tool's 'model' and 'profile' parameters are documented as 'restricted by server allowlist (disabled by default)' but no description of how to enable or what valid values are. This invites confusion and failed attempts.
The 'image' parameter accepts an array of strings but the description says 'Separate multiple paths with commas or repeat the flag,' which is CLI syntax, not JSON Schema semantics. The description is inconsistent with the parameter structure.
Tool annotations are correctly implemented (tool tests verify readOnlyHint and destructiveHint), but the definitions examined do not show idempotentHint declared. The codex tool with SESSION_ID parameter strongly suggests idempotent resumable sessions, but this is not formally declared.