Loading ToolBench
MCP server for CMS formulary data access with Code Mode execution, API catalog search, and data staging capabilities
This server exposes 5 code-execution tools with minimal parameter validation and vague descriptions. Tool names lack action verbs (execute, graphql_execute, sparql_execute are generic). Descriptions are lengthy but lack clarity on WHEN to use each tool vs. alternatives. Parameter schemas exist but lack enums, ranges, and validation constraints. No output schemas documented. Error handling is absent, code execution failures will surface raw V8 errors to the LLM. The 'code' parameter accepts arbitrary JavaScript with no sanitization, creating injection risks. Composition is poor: 5 similar tools (execute, graphql_execute, sparql_execute, search, catalog_search) overlap significantly, forcing LLMs to reason about which to call.
Search the API catalog (endpoints). Returns matching endpoints with full parameter docs. Use this to discover API capabilities before calling execute. Categories: (dynamic based on catalog) USAGE IN execute: - api.get(path, params) for GET, api.post(path, body, params) for POST - Path params like /lookup/{id} are auto-interpolated from params: api.get('/lookup/{id}', {id: 'ENSG...'}) - Remaining params become query string - Large responses (>100KB) are auto-staged: check result.__staged, return the staging info, use query_data to explore - Use limit/pagination params to control response size. Large datasets auto-stage for SQL queries.
Execute JavaScript code against the API with sandboxed V8 isolate. Code runs with: - searchSpec(query) — search the API catalog - listCategories() — list endpoint categories - getEndpoint(path) — get full endpoint docs - api.get(path, params) for GET, api.post(path, body, params) for POST - Path params like /lookup/{id} are auto-interpolated from params - Large responses (>100KB) are auto-staged: check result.__staged, return the staging info, use query_data to explore - Use limit/pagination params to control response size. Large datasets auto-stage for SQL queries.
Execute JavaScript code against GraphQL API with sandboxed V8 isolate. Code runs with: - gql.query(queryString, variables?) — GraphQL execution through host - schema.types(), schema.type(), schema.search() etc. — introspection helpers - db.stage(), db.queryStaged(), api.query() — staging helpers - console.log() capture
Search the API spec (OpenAPI mode). Write JavaScript code to search the OpenAPI spec, or use the legacy query/category arguments for keyword search. Available functions: - searchPaths(query, maxResults=10) — keyword search across paths, summaries, tags, parameters - listTags() — list all tags with operation counts - getOperation(idOrPath) — get full operation by operationId or path - describeOperation(idOrPath) — formatted documentation for an operation - searchSpec/query helpers are also available for backward compatibility inside execute() - spec — the full frozen OpenAPI spec object (spec.paths, spec.info, etc.) Use search to discover endpoints, then write code in execute to call them.
Tool names lack action verbs. 'execute', 'graphql_execute', 'sparql_execute' are generic and do not convey intent. LLMs cannot distinguish when to use execute vs. graphql_execute without reading lengthy descriptions.
No output schemas documented. Tools return arbitrary JavaScript results (code execution output, search results, etc.) with no structure. LLMs cannot plan downstream calls or extract required fields.
Code parameter accepts arbitrary JavaScript with no validation or sandboxing guarantees documented. Descriptions claim 'sandboxed V8 isolate' but provide no error handling, timeout limits, or injection prevention guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | 2026-07-28+ | v2 |
Execute JavaScript code against SPARQL endpoint. Code runs in sandboxed V8 isolate with: - sparql.query(queryString, opts?) — run SELECT/CONSTRUCT/DESCRIBE; returns array of plain { variable: value } rows - sparql.ask(queryString) — run ASK; returns boolean - sparql.raw(queryString, opts?) — return unparsed { head, results } envelope - prefixes — object of common ontology prefixes (rdfs, owl, xsd, skos, dcterms, void, obo, uberon, go, ncbigene, efo, obi, sio, up, ensembl, bgee) - prefixHeader — pre-built `PREFIX …` block to prepend to your queries - console logging (log, warn, error, info) — captured output
Five overlapping tools (execute, graphql_execute, sparql_execute, search, catalog_search) force LLMs to reason about which to call. No clear composition or tool-chaining guidance. Violates single-responsibility principle.
No error handling or recovery guidance. Code execution failures will surface raw V8 errors. LLMs have no actionable next steps when a query fails.