Loading ToolBench
OpenAI Codex CLI as an MCP server — use GPT-5.4 to build code from any MCP client
codex-mcp has well-structured tool definitions with complete input schemas and descriptions, but suffers from critical naming and composition issues. Tool names lack clear action verbs (codex_build, codex_review are vague about what 'codex' means to an LLM unfamiliar with the CLI). Descriptions are adequate (100-200 chars) but lack WHEN/WHY guidance and dependency hints. No output schemas documented, LLMs cannot plan downstream calls. Error handling is minimal (generic timeout/not-found messages). The server exposes a thin wrapper around an external CLI without abstracting complexity or guiding multi-step workflows. Parameters are well-typed but lack validation constraints (e.g., model enum, timeout bounds). No tool annotations (readOnlyHint, destructiveHint) despite clear risk levels (WRITE vs READ_ONLY).
Run OpenAI Codex (GPT-5.4) full-auto to build features, fix bugs, or write code in any directory. Codex reads the codebase, plans, writes files, runs tests. Best for: feature implementation, bug fixes, refactoring, test writing.
List Codex project memories for a directory. Memories are persistent notes Codex stores about a codebase.
Ask Codex a question with read-only access to a codebase. No file writes — pure analysis and reasoning. Good for: explaining code, architecture questions, debugging help.
Resume a previous Codex session by session ID. Useful for continuing interrupted builds or adding to previous work.
Run Codex code review on a repository. Analyzes code quality, bugs, security issues, and suggests improvements. Read-only.
List recent Codex sessions with their IDs, timestamps, and summaries.
Tool names lack clear action verbs. 'codex_build', 'codex_review', 'codex_query' are opaque to LLMs unfamiliar with the Codex CLI. Names should start with verbs like 'run_', 'execute_', 'analyze_' to signal intent. 'codex_build' could mean 'build codex' or 'use codex to build', ambiguous.
No output schemas documented. LLMs cannot infer what fields codex_build, codex_review, or codex_query return. Without documented return types, agents cannot plan chained calls or extract structured data. All tools return TextContent (raw strings), no structured results.
Tool annotations missing. codex_build and codex_resume are destructive (WRITE risk), but no destructiveHint in schema. codex_review, codex_query, codex_sessions, codex_skills, codex_memories are read-only, but no readOnlyHint. Agents cannot distinguish safe from risky tools without explicit hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
List or search Codex skills — reusable task templates stored in ~/.codex/skills/. Skills help Codex perform consistent, high-quality work on repeated task types.
Error handling is generic and non-actionable. Errors like 'codex failed (exit 1): <stderr>' or 'codex not found' do not guide recovery. No categorization (retryable vs fatal). No suggestions for next steps. LLMs receive raw subprocess output, not structured error guidance.
Parameter constraints missing. 'model' accepts free-form strings (default 'gpt-5.4', options 'o3, o4-mini' listed in description only). Should be an enum. 'timeout' is unbounded (default 600, no min/max). 'cwd' has no validation (path traversal risk). 'prompt' has no length limits.