Loading ToolBench
MCP server for ChatGPT automation and consultation via browser automation
codex_oracle has 9 tools with visible schemas and descriptions, but quality is inconsistent. Tool names lack clear verb-noun structure (consult, consult_prepare, consult_finalize are vague about what 'consult' means in context). Descriptions are present but generic (10-50 chars, well below the 50-200 char LLM-optimized baseline). Parameters have types and basic descriptions, but lack constraints (enums, ranges, format specs). No output schemas documented. Error handling is minimal, tools return generic toolError() responses with no recovery guidance. The server implements custom JSON-RPC parsing rather than a standard MCP library, increasing risk of spec drift.
Consult ChatGPT with a prompt and optional file attachments
Create a checkpoint of the current consultation session
Finalize a prepared consultation session
Prepare a consultation session without executing the prompt
Set up ChatGPT login in a private browser profile
Reattach to an existing ChatGPT browser session
Delete a stored consultation session
Tool names lack action verbs. 'consult', 'consult_prepare', 'consult_finalize' are vague, does 'consult' mean query, chat, execute, or something else? Names should be verb_noun (e.g., 'query_chatgpt', 'execute_prompt', 'prepare_session'). LLMs cannot infer intent from ambiguous names.
Descriptions are too short (10-50 chars) and lack context. 'Consult ChatGPT with a prompt and optional file attachments' (52 chars) does not explain WHEN to use this vs consult_prepare, what the output contains, or whether it modifies state. LLM-optimized descriptions are 50-200 chars with WHAT, WHEN, and WHAT-IT-RETURNS.
No output schemas documented. Tools return generic 'toolResult()' or 'toolError()' responses with no declared structure. LLMs cannot plan downstream calls or extract data without knowing what fields to expect. Every tool must document its return type.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | 2025-06-18+ | v2 |
List all stored consultation sessions
Perform a smoke check on the ChatGPT browser setup
Parameters lack constraints. 'preset' (string), 'model' (string), 'timeoutMs' (integer) have no enums, ranges, or format specs. LLMs will hallucinate invalid values. Add enums for preset/model, min/max for timeoutMs, and format specs for URLs.
Error handling is generic. All tools return 'toolError(msg)' with no recovery guidance. 'Invalid preset' tells the LLM nothing, should be 'Invalid preset: got "xyz", valid presets are: chatgpt-pro-heavy, chatgpt-standard, etc.' Errors must guide the agent's next step.