Loading ToolBench
MCP server for WikiPathways API integration with Code Mode execution, GraphQL/SPARQL support, and API catalog search
WikiPathways MCP exposes 6 tools with reasonable descriptions and parameter schemas. However, critical gaps exist: (1) Tool names lack clear verb-noun structure, '<prefix>_search' and '<prefix>_execute' are generic and overloaded across REST/GraphQL/SPARQL modes, violating single-responsibility principle. (2) Parameter descriptions are present but often verbose and example-heavy, inviting literal LLM reuse. (3) Output schemas are not documented, responses are described informally ('returns matching endpoints', 'returns array of rows') without structured field definitions. (4) Error handling is absent, no guidance on retryability, user-fixable vs fatal errors, or recovery paths. (5) Security: code execution via sandboxed V8 is high-risk; no mention of permission gates, audit logging, or rate limits. Baseline: 194 chars avg description (these are 150 - 300 chars, acceptable); 4 params avg (these range 1 - 3, good); but 0% have documented return schemas (should be 100%).
Execute JavaScript code against the API with sandboxed V8 isolate. Code runs with: searchSpec(query) / searchPaths(query) — search the API catalog/spec; listCategories() / listTags() — inspect tags/categories; getEndpoint(path) / getOperation(idOrPath) — get endpoint docs; api.get(path, params) for GET, api.post(path, body, params) for POST; Path params like /lookup/{id} are auto-interpolated from params; Remaining params become query string; Large responses (>100KB) are auto-staged: check result.__staged, return the staging info, use <prefix>_query_data to explore; console.log() capture; optional filesystem access (fs.readFile, fs.writeFile, fs.glob) when fsDoNamespace provided.
Execute JavaScript code against GraphQL endpoint with sandboxed V8 isolate. Code runs with: gql.query(queryString, variables?) — execute GraphQL queries; schema.types(), schema.type(), schema.search() — introspection helpers; db.stage(), db.queryStaged() — staging helpers; api.get/api.post (when restApiFetch provided) — REST capability; console.log() capture; optional filesystem access (fs.readFile, fs.writeFile, fs.glob) when fsDoNamespace provided. API keys never enter isolate — all HTTP through host's gqlFetch.
Execute JavaScript code against SPARQL endpoint with sandboxed V8 isolate. Code runs with: sparql.query(queryString, opts?) — run SELECT/CONSTRUCT/DESCRIBE; returns array of plain { variable: value } rows; sparql.ask(queryString) — run ASK; returns boolean; sparql.raw(queryString, opts?) — return unparsed { head, results } envelope; prefixes — object of common ontology prefixes (rdfs, owl, xsd, skos, dcterms, void, obo, uberon, go, ncbigene, efo, obi, sio, up, ensembl, bgee); prefixHeader — pre-built PREFIX block; db.stage(), db.queryStaged() — staging helpers; console logging; optional filesystem access when fsDoNamespace provided. Endpoint URL never enters isolate — all HTTP through host's sparqlFetch.
Tool names are generic and overloaded. Six tools share only two base names ('_search', '_execute') differentiated by parenthetical mode (REST/GraphQL/SPARQL). LLMs cannot disambiguate intent from names alone. Violates verb_noun convention and single-responsibility principle.
No documented output schemas. Descriptions use informal language ('returns matching endpoints', 'returns array of rows') without structured field definitions. LLMs cannot plan downstream calls or extract required data reliably.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | 2026-07-28+ | v2 |
Search the API catalog (legacy catalog mode). Returns matching endpoints with full parameter docs. Use this to discover API capabilities before calling <prefix>_execute. Categories are listed. USAGE IN <prefix>_execute: api.get(path, params) for GET, api.post(path, body, params) for POST. Path params like /lookup/{id} are auto-interpolated from params. Remaining params become query string. Large responses (>100KB) are auto-staged: check result.__staged, return the staging info, use <prefix>_query_data to explore. Use limit/pagination params to control response size.
Search GraphQL schema and discover operations. Write JavaScript code to search the schema, or use legacy query/category arguments. Available functions: searchTypes(query, maxResults=10) — keyword search across type names and descriptions; listTags() — list all tags with operation counts; getType(name) — get full type by name; describeType(name) — formatted documentation for a type; schema — the full frozen GraphQL introspection result.
Search the OpenAPI spec (new mode). Write JavaScript code to search the spec, or use legacy query/category arguments for keyword search. Available functions: searchPaths(query, maxResults=10) — keyword search across paths, summaries, tags, parameters; listTags() — list all tags with operation counts; getOperation(idOrPath) — get full operation by operationId or path; describeOperation(idOrPath) — formatted documentation for an operation; spec — the full frozen OpenAPI spec object (spec.paths, spec.info, etc.). Use <prefix>_search to discover endpoints, then write code in <prefix>_execute to call them.
No error handling guidance. Descriptions lack recovery paths, retryability classification, or actionable error messages. If code execution fails, LLM has no guidance on whether to retry, ask user, or abort.
Code execution tools (V8 sandbox) lack permission gates and audit logging. No mention of rate limits, user identity tracking, or scope declarations. High-risk operations require explicit security controls.
Parameter descriptions contain example values and verbose usage notes. LLMs tend to reuse examples literally (e.g. 'gene expression' as a literal query). Use enums and formal constraints instead.