Loading ToolBench
MCP server providing Code Mode execution tools for API discovery and execution, with support for REST APIs, GraphQL, and SPARQL endpoints
Server defines 4 tools with schemas visible in source. However, critical gaps undermine quality: (1) Tool naming is inconsistent, three tools share the name '{prefix}_execute' with different descriptions, violating single-responsibility and clarity principles. LLMs cannot distinguish between REST, GraphQL, and SPARQL variants by name alone. (2) Descriptions are verbose (200-400 chars) but lack actionable structure, they read as feature lists rather than 'WHAT does this do, WHEN to use it, WHAT does it return'. (3) Input schemas are present but minimal: 'code' param has a description, but no constraints on code length, timeout, or error recovery guidance. (4) Output schemas are completely undocumented, no indication of what these tools return, what fields to expect, or how to chain results. (5) No error handling guidance, descriptions mention 'console.log() capture' but do not explain what happens on syntax errors, timeouts, or sandbox violations. (6) Security: code execution in a sandbox is mentioned but no details on what's blocked, what's allowed, or how to handle malicious input.
Execute JavaScript code against the API. Code runs in a sandboxed V8 isolate with: searchSpec(query) / searchPaths(query) — search the API catalog or OpenAPI spec; listCategories() / listTags() — inspect tags/categories; getEndpoint(path) / getOperation(idOrPath) — get endpoint docs; api.get(path, params) for GET, api.post(path, body, params) for POST; Path params like /lookup/{id} are auto-interpolated from params; Remaining params become query string; Large responses (>100KB) are auto-staged: check result.__staged, return the staging info, use {prefix}_query_data to explore; console.log() capture; optional fs.* for virtual filesystem access.
Execute JavaScript code against the GraphQL API. Code runs in a sandboxed V8 isolate with: gql.query(queryString, variables?) — GraphQL execution through host; schema.types(), schema.type(), schema.search() etc. — introspection helpers; db.stage(), db.queryStaged(), api.query() — staging helpers; console.log() capture; optional fs.* for virtual filesystem access; optional api.get/api.post for hybrid REST capability.
Execute JavaScript code against the SPARQL endpoint. Code runs in a sandboxed V8 isolate with: sparql.query(queryString, opts?) — run SELECT/CONSTRUCT/DESCRIBE; returns array of plain { variable: value } rows; sparql.ask(queryString) — run ASK; returns boolean; sparql.raw(queryString, opts?) — return unparsed { head, results } envelope; prefixes — object of common ontology prefixes (rdfs, owl, xsd, skos, dcterms, void, obo, uberon, go, ncbigene, efo, obi, sio, up, ensembl, bgee); prefixHeader — pre-built PREFIX block to prepend to queries; console logging (log, warn, error, info) — captured output; optional fs.* for virtual filesystem access. Large SPARQL responses (>30KB) are auto-staged into SQLite.
Search the API catalog (catalog mode) or OpenAPI spec (OpenAPI mode). Returns matching endpoints with full parameter docs. Use this to discover API capabilities before calling {prefix}_execute. Categories are listed. USAGE IN {prefix}_execute: api.get(path, params) for GET, api.post(path, body, params) for POST. Path params like /lookup/{id} are auto-interpolated from params. Remaining params become query string. Large responses (>100KB) are auto-staged: check result.__staged, return the staging info, use {prefix}_query_data to explore. Use limit/pagination params to control response size.
Three tools named '{prefix}_execute' with different behaviors (REST, GraphQL, SPARQL). LLMs cannot distinguish them by name; violates single-responsibility and clarity principles. Should be '{prefix}_execute_rest', '{prefix}_execute_graphql', '{prefix}_execute_sparql'.
No output schemas documented for any tool. Descriptions mention 'returns array', 'returns boolean', 'auto-staged' but do not specify field names, types, or structure. LLMs cannot plan downstream calls or extract data reliably.
Input parameter 'code' lacks constraints: no max length, no timeout guidance, no error recovery hints. Descriptions mention sandbox features (searchPaths, api.get, gql.query) but do not explain what happens on syntax errors, infinite loops, or permission violations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 34 | 2026-07-28+ | v2 |
Descriptions are feature-lists (200-400 chars) rather than actionable guidance. Missing: WHEN to use each tool, WHAT it returns, HOW to handle errors. E.g., '{prefix}_search' description does not explain when to call it vs '{prefix}_execute'.
No error handling guidance. Descriptions mention 'console.log() capture' and 'auto-staged' but do not explain failure modes, recovery steps, or what to do if code execution fails. LLMs have no guidance on retryability or next steps.