Loading ToolBench
MCP server providing tools for API discovery, code execution, GraphQL queries, and SPARQL endpoint access with auto-staging and citation tracking
Server defines 5 tools with detailed descriptions and complex parameter schemas. However, critical gaps exist: (1) Tool names lack clear verb-noun structure, '{prefix}_search' and '{prefix}_execute' are generic and repeated across different APIs (REST, GraphQL, SPARQL), making them ambiguous for LLM selection. (2) Parameter descriptions are verbose but lack actionable constraints (enums, ranges, patterns). (3) Output schemas are not documented, descriptions mention auto-staging and SQLite but do not specify the actual response structure. (4) Error handling guidance is minimal. (5) Security considerations (secret injection, permission gates) are not visible in tool definitions.
Execute JavaScript code against the GraphQL API with sandboxed V8 isolate. Code runs with: gql.query(queryString, variables?) — GraphQL execution; schema.types(), schema.type(), schema.search() — introspection helpers; db.stage(), db.queryStaged() — staging helpers; api.get/api.post (when restApiFetch provided) — REST capability override; console.log() capture; fs.* for virtual filesystem (when fsDoNamespace provided). Large GraphQL responses (>30KB) are auto-staged into SQLite. When staging occurs, gql.query returns {__staged: true, data_access_id, schema, tables_created, total_rows, message}. Use api.query(data_access_id, sql) in-band or return the staging object for {prefix}_query_data.
Execute JavaScript code against the API with sandboxed V8 isolate. Code runs with: searchSpec(query) / searchPaths(query) — search the API catalog or OpenAPI spec; listCategories() / listTags() — inspect tags/categories; getEndpoint(path) / getOperation(idOrPath) — get endpoint docs; describeEndpoint(path, method?) / describeOperation(idOrPath) — format endpoint docs; spec — full frozen OpenAPI spec object; api.get(path, params) for GET, api.post(path, body, params) for POST; api.getAll(path, params) for exhaustive fetch; console.log() capture; fs.* for virtual filesystem (when fsDoNamespace provided); db.stage(), db.queryStaged() for auto-staging. Large responses (>100KB) are auto-staged into SQLite. Path params like /lookup/{id} are auto-interpolated from params. Remaining params become query string.
Execute JavaScript code against the SPARQL endpoint. Code runs in a sandboxed V8 isolate with: sparql.query(queryString, opts?) — run SELECT/CONSTRUCT/DESCRIBE; returns array of plain { variable: value } rows; sparql.ask(queryString) — run ASK; returns boolean; sparql.raw(queryString, opts?) — return unparsed { head, results } envelope; prefixes — object of common ontology prefixes (rdfs, owl, xsd, skos, dcterms, void, obo, uberon, go, ncbigene, efo, obi, sio, up, ensembl, bgee); prefixHeader — pre-built `PREFIX …` block to prepend to queries; console logging (log, warn, error, info) — captured output; fs.* for virtual filesystem (when fsDoNamespace provided). Large SPARQL responses (>30KB) are auto-staged into SQLite. When staging occurs, sparql.query returns {__staged: true, data_access_id, schema, tables_created, total_rows, message}. Use api.query(data_access_id, sql) in-band or return the staging object for {prefix}_query_data. Start with small SELECT or ASK queries and add LIMIT early.
Generic, repeated tool names lack verb-noun clarity. '{prefix}_search' and '{prefix}_execute' appear 3+ times across REST, GraphQL, and SPARQL APIs. LLMs cannot distinguish which API each tool targets without reading full descriptions. Should be 'search_rest_catalog', 'execute_rest_code', 'search_graphql_schema', 'execute_graphql_code', 'execute_sparql_query'.
Output schemas not documented. Descriptions mention '__staged', 'data_access_id', 'schema', 'tables_created', 'total_rows' but do not specify the actual response object structure, field types, or when staging occurs. LLMs cannot plan downstream calls without knowing what fields to expect.
Parameter constraints lack formal specification. 'max_results' accepts 'number' but description says 'default 10, max 25', no enum, min/max bounds, or validation rules in schema. 'code' parameter accepts arbitrary JavaScript with no length limit, format validation, or injection guards documented.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 39 | 2026-07-28+ | v2 |
Search the API catalog (catalog mode) or OpenAPI spec (OpenAPI mode). Returns matching endpoints with full parameter docs. Use this to discover API capabilities before calling {prefix}_execute. Categories are listed. USAGE IN {prefix}_execute: api.get(path, params) for GET, api.post(path, body, params) for POST. Path params like /lookup/{id} are auto-interpolated from params. Remaining params become query string. Large responses (>100KB) are auto-staged: check result.__staged, return the staging info, use {prefix}_query_data to explore. Use limit/pagination params to control response size.
Search the GraphQL API schema. Write JavaScript code to search the schema, or use legacy query/category arguments for keyword search. Available functions: searchTypes(query, maxResults=10) — keyword search across type names and descriptions; listTags() — list all tags with operation counts; getType(name) — get full type by name; describeType(name) — formatted documentation for a type; schema — the full frozen GraphQL schema object. Use {prefix}_search to discover types and fields, then write code in {prefix}_execute to query them.
Error handling guidance absent. Descriptions do not explain what errors can occur, how to recover, or what the LLM should do if a query fails. No mention of timeout behavior, rate limits, or partial failure modes.
Security model not visible. Tools accept arbitrary JavaScript code execution in sandboxed V8 isolates. No documentation of what code can access, what is blocked, or how secrets are injected. No mention of permission gates, audit logging, or rate limits.