Loading ToolBench
MCP server for executing code against APIs (REST, GraphQL, SPARQL) with sandboxed V8 isolates, auto-staging of large responses, and schema discovery
Server defines 5 tools with reasonable descriptions and schemas. All tools have descriptions (50-300 chars, within baseline 34-392 range). Input schemas present with Zod types and descriptions. However, output schemas are not documented, LLMs cannot predict return structure. Tool names are action-oriented (search, execute, graphql_execute, sparql_execute) but lack verb_noun clarity for some (catalog_search vs search). Error handling guidance is minimal. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all being read-only operations. Parameter descriptions are present but some lack format/constraint details (e.g., 'code' param in execute lacks length/complexity bounds).
Search the API catalog (legacy mode). Returns matching endpoints with full parameter docs. Use this to discover API capabilities before calling execute. Categories are listed. USAGE IN execute: api.get(path, params) for GET, api.post(path, body, params) for POST. Path params like /lookup/{id} are auto-interpolated from params. Remaining params become query string. Large responses (>100KB) are auto-staged: check result.__staged, return the staging info, use query_data to explore. Use limit/pagination params to control response size.
Execute JavaScript code against an API with sandboxed V8 isolate. Code runs with: searchSpec(query) / searchPaths(query) — search the API catalog/spec; listCategories() / listTags() — inspect tags/categories; getEndpoint(path) / getOperation(idOrPath) — get endpoint docs; api.get(path, params) for GET, api.post(path, body, params) for POST; Path params like /lookup/{id} are auto-interpolated from params; Remaining params become query string; Large responses (>100KB) are auto-staged: check result.__staged, return the staging info, use query_data to explore; console.log() capture; optional filesystem access (fs.readFile, fs.writeFile, fs.glob)
Execute JavaScript code against a GraphQL endpoint with sandboxed V8 isolate. Code runs with: gql.query(queryString, variables?) — GraphQL execution through host; schema.types(), schema.type(), schema.search() etc. — introspection helpers; db.stage(), db.queryStaged(), api.query() — staging helpers; console.log() capture; optional filesystem access; optional REST API access (api.get/api.post) when restApiFetch is configured
Search the API spec (OpenAPI mode). Write JavaScript code to search the OpenAPI spec, or use legacy query/category arguments for keyword search. Available functions: searchPaths(query, maxResults=10) — keyword search across paths, summaries, tags, parameters; listTags() — list all tags with operation counts; getOperation(idOrPath) — get full operation by operationId or path; describeOperation(idOrPath) — formatted documentation for an operation; spec — the full frozen OpenAPI spec object. Use search to discover endpoints, then write code in execute to call them.
Output schemas not documented. LLMs cannot predict return structure (fields, types, pagination). Rubric baseline: 100% of A+ tools have documented return types.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All 5 tools are read-only but lack explicit annotation. Agents cannot distinguish safe-to-retry operations from destructive ones.
Parameter 'code' in execute/graphql_execute/sparql_execute lacks constraints. No max length, complexity bounds, or timeout guidance. Unbounded code strings invite resource exhaustion.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
Execute JavaScript code against a SPARQL endpoint with sandboxed V8 isolate. Code runs with: sparql.query(queryString, opts?) — run SELECT/CONSTRUCT/DESCRIBE; returns array of plain { variable: value } rows; sparql.ask(queryString) — run ASK; returns boolean; sparql.raw(queryString, opts?) — return unparsed { head, results } envelope; prefixes — object of common ontology prefixes (rdfs, owl, xsd, skos, dcterms, void, obo, uberon, go, ncbigene, efo, obi, sio, up, ensembl, bgee); prefixHeader — pre-built PREFIX block to prepend to queries; console logging (log, warn, error, info) — captured output; optional filesystem access. Large SPARQL responses (>30KB) are auto-staged into SQLite.
Error handling guidance minimal. Descriptions mention staging and auto-interpolation but do not explain failure modes (invalid code, timeout, API error) or recovery steps.
Naming ambiguity: 'search' vs 'catalog_search' both perform keyword search but on different backends (OpenAPI spec vs static catalog). LLMs may conflate them. Descriptions clarify but names should be more distinct.