Loading ToolBench
MCP server that runs codex reviews and allowlisted commands on the host machine, with background job support for long-running reviews
Three tools with complete schemas and descriptions. Tool naming follows verb_noun pattern (codex_review_start, codex_review_status, run_command). Descriptions are detailed (150-250 chars) and explain WHEN to use each tool. Parameters have types and descriptions. However, output schemas are not documented, LLMs cannot predict response structure. Error handling is basic (returns error string, not structured guidance). run_command lacks critical security documentation around allowlist enforcement. No tool annotations (readOnlyHint/destructiveHint) despite clear risk profiles.
Start a read-only codex review of changes in a host project folder. The review runs in the background on the host and this call returns a job_id immediately; poll codex_review_status to get the result. (Reviews often take several minutes — longer than the MCP client's ~180s per-call limit — which is why they run as background jobs.) Uses 'codex review', which computes the diff itself and does not modify files.
Get the status/result of a background review started with codex_review_start. Blocks up to ~50s waiting for completion, then returns either the finished review or a 'running' notice — keep calling it with the same job_id until it completes. Finished results stay available for ~30 minutes.
Run an allowlisted command (argv, no shell) in a host folder.
Output schemas not documented. LLMs cannot predict response structure (fields, types, pagination). codex_review_start returns job_id and status message; codex_review_status returns review text or 'running' notice; run_command returns command output. None of these are formally specified.
run_command description lacks security context. Does not explain what 'allowlist' means, how to discover allowed commands, or what happens if a command is not allowlisted. Users/LLMs cannot safely use this tool without external documentation.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint). codex_review_start and codex_review_status are read-only; run_command is destructive (can modify host state). LLMs cannot infer safety properties without explicit hints.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | <=2025-11-25 | v2 |
Error responses are unstructured strings. 'unknown job_id' or 'Invalid priority' are human-readable but do not guide LLM recovery. No categorization (retryable vs user-fixable vs fatal) or actionable next steps.
run_command parameter 'command' is a bare string with no enum or allowlist documentation. LLMs cannot discover valid commands or validate input before calling. Description says 'must be in the server allowlist' but does not explain how to check.