Loading ToolBench
MCP server for OpenPayments API integration with Code Mode execution, GraphQL/SPARQL support, and API catalog search
This server exposes 5 code-execution tools (catalog_search, execute, search, graphql_execute, sparql_execute) with substantial descriptions but critical schema and composition gaps. Tool names lack clear verb-noun structure (execute, search are generic). Input schemas are visible but incomplete: most parameters lack type constraints, enums, or validation guidance. Descriptions are verbose (200-400 chars) but bury actionable constraints. No output schemas documented. Error handling is absent, no recovery guidance, no categorization. The execute/graphql_execute/sparql_execute tools conflate multiple concerns (discovery + execution + staging) into single tools, violating single-responsibility. No tool annotations (readOnlyHint, idempotentHint). Composition is broken: tools return staging info but no clear guidance on how to use query_data or workspace parameters across calls.
Search the API catalog (legacy mode). Returns matching endpoints with full parameter docs. Use this to discover API capabilities before calling execute. Categories are listed. USAGE IN execute: api.get(path, params) for GET, api.post(path, body, params) for POST. Path params like /lookup/{id} are auto-interpolated from params. Remaining params become query string. Large responses (>100KB) are auto-staged: check result.__staged, return the staging info, use query_data to explore. Use limit/pagination params to control response size.
Execute JavaScript code against the API with sandboxed V8 isolate. Code runs with: searchSpec(query) / searchPaths(query) — search the API catalog/spec; listCategories() / listTags() — inspect tags/categories; getEndpoint(path) / getOperation(idOrPath) — get endpoint docs; describeEndpoint(path, method?) / describeOperation(idOrPath) — format endpoint docs; spec — full frozen OpenAPI spec object; api.get(path, params) for GET, api.post(path, body, params) for POST; Path params like /lookup/{id} are auto-interpolated from params; Remaining params become query string; Large responses (>100KB) are auto-staged: check result.__staged, return the staging info, use query_data to explore; console.log() capture; optional fs.* for virtual filesystem.
Execute JavaScript code against GraphQL endpoint with sandboxed V8 isolate. Code runs with: gql.query(queryString, variables?) — GraphQL execution through host; schema.types(), schema.type(), schema.search() etc. — introspection helpers; db.stage(), db.queryStaged(), api.query() — staging helpers; console.log() capture; optional fs.* for virtual filesystem; optional api.get/api.post for hybrid REST capability.
Search the OpenAPI spec (new mode). Write JavaScript code to search the OpenAPI spec, or use legacy query/category arguments for keyword search. Available functions: searchPaths(query, maxResults=10) — keyword search across paths, summaries, tags, parameters; listTags() — list all tags with operation counts; getOperation(idOrPath) — get full operation by operationId or path; describeOperation(idOrPath) — formatted documentation for an operation; searchSpec/query helpers are also available for backward compatibility inside execute(); spec — the full frozen OpenAPI spec object (spec.paths, spec.info, etc.). Use search to discover endpoints, then write code in execute to call them.
Generic tool names lack action verbs. 'execute' and 'search' are ambiguous, does execute run code, API calls, or queries? Does search query the catalog or spec? Names should be verb_noun: run_javascript_code, query_openapi_spec, execute_graphql_query.
Input schemas lack type constraints and enums. 'code' parameter accepts any string with no length limit, format validation, or examples of valid syntax. 'workspace' is undocumented, what format? Required? How does it relate to staging? Parameters need explicit constraints (max length, allowed characters, enum values).
No output schemas documented. Tools return complex objects (result.__staged, staging info, query results) but LLMs cannot plan downstream calls without knowing the response structure. Document: what fields does result contain? What is __staged? What does query_data return?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 34 | 2026-07-28+ | v2 |
Execute JavaScript code against SPARQL endpoint with sandboxed V8 isolate. Code runs with: sparql.query(queryString, opts?) — run SELECT/CONSTRUCT/DESCRIBE; returns array of plain { variable: value } rows; sparql.ask(queryString) — run ASK; returns boolean; sparql.raw(queryString, opts?) — return unparsed { head, results } envelope; prefixes — object of common ontology prefixes (rdfs, owl, xsd, skos, dcterms, void, obo, uberon, go, ncbigene, efo, obi, sio, up, ensembl, bgee); prefixHeader — pre-built PREFIX block to prepend to queries; console logging (log, warn, error, info) — captured output; optional fs.* for virtual filesystem. Large SPARQL responses (>30KB) are auto-staged into SQLite.
No error handling or recovery guidance. Descriptions mention 'auto-staged' and 'check result.__staged' but provide no error messages, validation failures, or what to do if code execution fails. Error responses must tell LLMs what to do next (retry, ask user, abort).
Tool composition is broken. execute, graphql_execute, and sparql_execute each accept 'code' and 'workspace' but descriptions do not explain how workspace IDs are created, shared, or referenced across calls. Staging mechanism is opaque, no guidance on when to use query_data vs direct result access.