Loading ToolBench
MCP server providing Code Mode execution tools for API discovery and execution, GraphQL introspection, and SPARQL endpoint querying with sandboxed V8 isolate execution
Server defines 5 tools with reasonable descriptions and input schemas. All tools have descriptions (50-300 chars, within baseline 34-392 range). Input schemas use Zod with type definitions and parameter descriptions. However, output schemas are not documented, LLMs cannot predict return structure. Parameter descriptions are present but sometimes vague (e.g., 'JavaScript code' lacks constraints on what functions/APIs are available). No error handling guidance, no enum constraints on free-form inputs, and no documentation of staging/pagination behavior in structured output. Tools are well-named (verb_noun pattern: catalog_search, execute, search, graphql_execute, sparql_execute) but lack actionable error recovery hints.
Search the API catalog (legacy mode). Returns matching endpoints with full parameter docs. Use this to discover API capabilities before calling execute. Categories are listed. USAGE IN execute: api.get(path, params) for GET, api.post(path, body, params) for POST. Path params like /lookup/{id} are auto-interpolated from params. Remaining params become query string. Large responses (>100KB) are auto-staged: check result.__staged, return the staging info, use query_data to explore. Use limit/pagination params to control response size.
Execute JavaScript code against an API catalog or OpenAPI spec in a sandboxed V8 isolate. Code runs with api.get/api.post wrappers, catalog search helpers (searchSpec, listCategories, getEndpoint), console.log capture, and optional filesystem access. Large responses (>100KB) are auto-staged into SQLite. API keys never enter the isolate — all HTTP goes through the host's apiFetch.
Execute JavaScript code against a GraphQL endpoint in a sandboxed V8 isolate. Code runs with gql.query(queryString, variables?) for GraphQL execution, schema.types()/schema.type()/schema.search() for introspection helpers, db.stage()/db.queryStaged() for staging, and optional REST api.get/api.post when a restApiFetch is configured. Large responses (>30KB) are auto-staged into SQLite. API keys never enter the isolate — all HTTP goes through the host's gqlFetch.
Search the OpenAPI spec or API catalog. Write JavaScript code to search the spec, or use legacy query/category arguments for keyword search. Available functions: searchPaths(query, maxResults=10) — keyword search across paths, summaries, tags, parameters; listTags() — list all tags with operation counts; getOperation(idOrPath) — get full operation by operationId or path; describeOperation(idOrPath) — formatted documentation for an operation; spec — the full frozen OpenAPI spec object.
Output schemas not documented. LLMs cannot predict return structure (fields, types, pagination). Descriptions mention '__staged' and 'staging info' but no formal schema for these fields.
Code parameters ('code' in execute, graphql_execute, sparql_execute, search) accept free-form JavaScript but lack constraints on available APIs, error handling, or timeout behavior. Descriptions mention 'api.get', 'gql.query', 'sparql.query' but do not specify what happens on invalid code or missing functions.
No error handling guidance. Descriptions do not explain what errors are retryable, what indicates user misconfiguration, or how to recover from staging failures, API errors, or code execution timeouts.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
Execute JavaScript code against a SPARQL endpoint in a sandboxed V8 isolate. Code runs with sparql.query(queryString, opts?) for SELECT/CONSTRUCT/DESCRIBE, sparql.ask(queryString) for ASK queries, sparql.raw(queryString, opts?) for unparsed JSON envelope, prefixes object with common ontology prefixes, prefixHeader pre-built PREFIX block, and optional filesystem access. Large SPARQL responses (>30KB) are auto-staged into SQLite. The endpoint URL never enters the isolate — all HTTP goes through the host's sparqlFetch.
Pagination and result limits mentioned in descriptions but not formalized in schema. 'max_results' is optional in catalog_search but absent from other tools. No 'limit', 'offset', or 'next_cursor' parameters documented.
Workspace parameter in execute and graphql_execute is optional and underdocumented. No description of what workspace IDs are valid, how they persist, or what 'shared staging' means to the LLM.