Loading ToolBench
MCP server that provides access to ChatGPT Desktop App via UI automation. Allows querying ChatGPT and listing available models through the Model Context Protocol.
Two tools with minimal descriptions and incomplete schemas. 'search' has a 95-char description (below baseline 194 chars) and parameters lack type information in visible schema. 'list_models' has only 34 chars (below 10-char minimum for meaningful descriptions). Neither tool has documented output schemas. No error handling guidance, no parameter constraints (enums), no recovery paths. Schemas appear inferred rather than explicitly registered with full JSON Schema definitions.
List all available ChatGPT models.
Ask ChatGPT a question via the ChatGPT Desktop App. Returns the AI response.
Missing output schemas for both tools. LLMs cannot plan downstream calls or extract required fields without knowing what 'search' returns or what fields 'list_models' provides.
list_models description is 34 characters ('List all available ChatGPT models.'), below the 10-char minimum for meaningful context. LLMs cannot determine when to call this tool vs. relying on cached knowledge.
search parameter 'model' lacks description. LLMs cannot infer whether to pass a full model name, a prefix, or an ID. The hint 'Call list_models to see available models' is buried in the parameter description, not in the tool description.
No error handling guidance. If ChatGPT Desktop App is not running or a query fails, the tool provides no recovery path. LLMs cannot self-correct or suggest alternatives.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | 2026-07-28+ | v2 |
web_search parameter lacks constraints. No description of what 'true' vs 'false' does, no guidance on when to enable it, no mention of latency or cost implications.