Loading ToolBench
Code Mode MCP server for Apple Search Ads — 2 tools, hand-built spec (Apple does not publish OpenAPI for ASA)
Both tools have severe definition quality issues. The 'search' and 'execute' tools expose a dangerous code-execution interface with minimal guardrails. Descriptions are verbose (500+ chars) but lack clarity on WHEN to use each tool and what distinguishes them. Input schemas are minimal (single 'code' string parameter with no validation, constraints, or format guidance). No output schemas documented. No error handling guidance. The descriptions read as API documentation rather than LLM-optimized tool selection prompts. No parameter descriptions beyond the code field itself. No mention of security implications, rate limits, or failure modes.
Write JavaScript to call the Apple Search Ads API. Auth + org context auto-injected. Available globals: - `api` — authenticated client for ASA. Usage: const r = await api.request({ method: 'GET' | 'POST' | 'PUT' | 'DELETE', path: '/campaigns', // from search results params: { limit: 1000 }, // query string (optional) body: { ... }, // POST/PUT body (optional) orgId: 'YOUR_NON_US_ORG_ID', // override org for Non-US (optional, default is main US org) }); Reports endpoints return: r.data.reportingDataResponse.row[] Each row: { metadata: {...}, total: {...}, granularity: [{date, localSpend, tapInstalls, ...}] } Errors: Apple returns { data: null, error: { errors: [{ messageCode, message, field }] } } The client throws them as "ASA API 404: RESOURCE_NOT_FOUND: ..." strings. Example — pull Brand campaign 14d keyword report: const rep = await api.request({ method: 'POST', path: '/reports/campaigns/YOUR_CAMPAIGN_ID/keywords', body: { startTime: '2026-04-10', endTime: '2026-04-23', granularity: 'DAILY', selector: { orderBy: [{ field: 'localSpend', sortOrder: 'DESCENDING' }], pagination: { offset: 0, limit: 1000 } }, returnRowTotals: true, returnGrandTotals: true } }); return rep.data.reportingDataResponse.row.map(r => ({ kw: r.metadata.keyword, spend: r.total.localSpend.amount, installs: r.total.tapInstalls }));
Write JavaScript to explore the Apple Search Ads API specification (${this.spec.pathCount} endpoints, ${this.spec.schemaCount} schemas, hand-built from ${this.spec.info.version}). Available globals: - `spec` — Object with API endpoints. Structure: spec.paths['/campaigns'].get ⚠️ NOT a complete Apple ASA API spec. Covers the endpoints Trial and Error Inc. actively uses. Expand src/spec/openapi.json if you need more. Apple does not publish an official OpenAPI for ASA. How to use: - List endpoints: Object.keys(spec.paths) - Filter by keyword: Object.entries(spec.paths).filter(([p]) => p.includes('keyword')) - Get details: spec.paths['/campaigns/{campaignId}/adgroups/{adGroupId}/targetingkeywords/bulk'].put - Check parameters: ... .parameters - Check request body: ... .requestBody.schema - Check response: ... .responses['200'].schema Return findings as a value or use console.log(). Critical gotchas already documented in spec descriptions: - POST /targetingkeywords silently overrides bidAmount to $1 on create — always follow with bulk PUT - PUT /targetingkeywords/{id} does NOT exist (404) — use bulk endpoint even for single updates - Negative keyword batch limit: 5 at a time (Feb 13 2026 incident protocol)
No input validation or constraints on 'code' parameter. LLMs can pass arbitrary JavaScript, including infinite loops, memory exhaustion, or API abuse. No mention of sandbox limits, execution timeout, or resource caps.
No output schema documented for either tool. LLMs cannot predict what 'search' returns (raw spec object? formatted JSON? console.log output?) or what 'execute' returns (API response? parsed data? error object?). Breaks downstream tool chaining.
Tool names 'search' and 'execute' are generic and ambiguous. 'search' could mean search the API spec, search the API itself, or search documentation. 'execute' could mean execute code, execute API calls, or execute a workflow. No verb_noun clarity.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 38 | 2026-07-28+ | v2 |
Descriptions are 500+ characters (baseline 194 avg, p90=392). Verbose API docs buried in tool descriptions waste tokens and obscure the core purpose. LLMs struggle to decide between 'search' and 'execute' when both descriptions are lengthy tutorials.
No error handling guidance. If code execution fails (syntax error, timeout, API error), the tool returns a raw error string. No recovery hints, no categorization (retryable vs fatal), no actionable next steps for the LLM.