Loading ToolBench
A client for executing Python code on a remote MCP server with OAuth authentication
This MCP server has a single tool 'execute_python' with critical deficiencies. The tool has a schema (code parameter with type string), but lacks essential quality markers across naming, description, and error handling. The source shows only a client implementation calling a remote MCP server, there is no server-side tool registration visible in the provided code. The actual tool definition (if it exists) is on the remote server at 'https://idp.objectgraph.com/mcp/', not in this repo. Without visibility into the server-side implementation, tool scores are capped at 50 per the rubric. Additionally, the single visible tool lacks proper description, no parameter descriptions beyond trivial type info, no documented output schema, and critically, no error handling or recovery guidance for an irreversible operation (arbitrary Python code execution).
No visible server-side tool definition in repository. This is a client implementation only.
Tool 'execute_python' has no description in visible code. A tool must have a non-empty description (10-1024 chars) so LLMs can determine when to select it. Without one, the LLM cannot reason about whether this tool is appropriate for a given task.
Parameter 'code' has a trivial description ('Python code to execute') under 20 characters. The description does not explain format expectations, constraints (memory limits, execution timeout, available libraries), or examples of what constitutes valid code.
Irreversible operation ('execute_python' can modify system state, access files, execute arbitrary code) has no dry-run, confirmation step, or undo mechanism. Per pattern:confirmation-request, destructive tools should support user confirmation before execution.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 37 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 29 | - | v1 |
No tool annotations visible. Tools executing arbitrary code or modifying state should declare destructiveHint=true and/or idempotentHint=false so LLMs understand the risk profile and avoid accidental misuse.
No output schema documented. The tool's return type is unknown. LLMs need to know what fields to expect (e.g., stdout, stderr, exit_code, result, error) to plan downstream operations and extract relevant data.
No error handling or recovery guidance. If Python execution fails (syntax error, timeout, permission denied), the tool response must guide the LLM on what to do next. A stack trace or generic error tells the agent nothing actionable.
Arbitrary code execution is a critical security risk. No constraints documented: no timeout, memory limits, accessible paths, banned imports, or permission checks. The tool description must declare what execution environment is available and what is forbidden.
No idempotency guarantees. Executing the same code twice may produce different results (e.g., file writes, network calls, time-dependent operations). Per pattern:idempotent-operation, tools should be designed for safe retry or document when retry is unsafe.