Loading ToolBench
Code-mode MCP server for local Ubiquiti consoles.
Single tool 'execute_python' has a well-structured description (1200+ chars) that explains the sandboxed environment, available imports, API patterns, and includes examples. Input schema is present with proper type and description. However, the tool conflates multiple concerns (inspection, filtering, parallel execution) into one generic code-execution interface. The description is verbose and tutorial-like rather than concise (ideal 50-200 chars per pattern:tool-description). No output schema is documented, LLMs cannot predict what fields to expect from arbitrary Python execution. Error handling relies on RuntimeError propagation with no recovery guidance. The tool accepts arbitrary Python code, which is inherently high-risk and lacks input validation or safety constraints typical of production tools.
CODE MODE: execute Python to inspect the local Ubiquiti network. ENVIRONMENT - This is a sandboxed Python interpreter, not CPython. No filesystem, no environment, no network, no eval/exec: the methods below are the only way out. - The methods listed below are injected as globals; call them directly. Do not import or construct anything. - Importable: json, re, math, collections, itertools, datetime, dataclasses, typing, asyncio, unicodedata (plus os, sys and pathlib, though file and environment access is denied). Nothing else exists, so functools, statistics, ipaddress, enum, copy, random and time are unavailable. - Both the value of the last expression and anything you print() come back. Ending on a bare collection dumps all of it, so prefer printing the fields you need. - Each call runs in a fresh namespace, so nothing persists between calls. - API failures raise RuntimeError carrying the console's message; subclasses do not survive the sandbox boundary, so catch RuntimeError if you must catch anything. You get the traceback either way, so prefer letting them propagate. - This build is read-only: every method below is a GET. The API's create/update/delete endpoints are not exposed. - There is no clock, so datetime.now() and date.today() are unavailable. Call now() for the current UTC time as an ISO 8601 string: datetime.datetime.fromisoformat(now()).
Tool name 'execute_python' is generic and does not follow verb_noun pattern. 'execute' is vague, does not convey that this inspects a Ubiquiti network. Better: 'inspect_ubiquiti_network' or 'query_unifi_devices'.
No output schema documented. LLMs cannot predict return type or structure. Tool description says 'Both the value of the last expression and anything you print() come back' but does not specify JSON structure, field names, or types. Agents cannot plan downstream operations.
Description is 1200+ characters, far exceeds the 50-200 char baseline for A+ tools. Verbose tutorial-style text wastes tokens and buries key details. Should be: 'Execute Python code to query Ubiquiti network devices, clients, and statistics. Code runs in a sandboxed interpreter with access to describe(), get_devices(), get_clients(), and parallel() methods.'
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | <=2025-11-25 | v2 |
Accepts arbitrary Python code with no input validation, sanitization, or constraints. Although sandboxed, this violates pattern:tool-gateway (treat agent input as untrusted). No rate limiting, no confirmation step for destructive operations, no audit trail of what code was executed.
Error handling relies on RuntimeError propagation with no recovery guidance. Description says 'API failures raise RuntimeError' but does not tell LLM what to do next. Should return structured errors like 'Device not found. Available devices: [list]. Try describe("getDevices") to see schema.'