Loading ToolBench
MCP server for eQTL Catalogue API discovery and execution with Code Mode support
This server exposes 5 tools with factory patterns for code execution across multiple query languages (Catalog, OpenAPI, GraphQL, SPARQL). While schemas are present and mostly typed, descriptions are verbose and lack LLM-optimized clarity. Critical gaps: no error handling guidance, no output schema documentation, no parameter validation rules, and tool names are generic factory names rather than action verbs. The 'createExecuteTool' and similar names obscure what the tools actually do. Descriptions exceed 1024 chars in several cases, violating token efficiency. No evidence of permission gates, audit trails, or secret injection patterns despite executing arbitrary user code in sandboxed isolates.
Search the API catalog (legacy). The tool accepts query/category/max_results parameters and performs keyword-based search over the static ApiCatalog. Returns matching endpoints with full parameter docs. Use this to discover API capabilities before calling execute.
Execute tool factory — creates a <prefix>_execute tool for Code Mode. Uses DynamicWorkerExecutor to run user code in a sandboxed V8 isolate with API proxy, catalog search helpers, and console.log() capture.
GraphQL execute tool factory — creates a <prefix>_execute tool for GraphQL Code Mode. Uses DynamicWorkerExecutor to run user code in a sandboxed V8 isolate with gql.query(), schema introspection helpers, and optional REST API support.
Create a search tool in OpenAPI mode. The tool accepts a code parameter — agent-written JavaScript that runs with the full resolved OpenAPI spec and helper functions (searchPaths, listTags, getOperation, describeOperation) available.
SPARQL execute tool factory — creates a <prefix>_execute tool for SPARQL Code Mode. The isolate gets sparql.query(), sparql.ask(), sparql.raw(), common ontology prefixes, and staging helpers.
Tool names are factory function names ('createExecuteTool', 'createGraphqlExecuteTool') rather than action verbs describing what the tool does. LLMs cannot infer intent from 'create*', they need 'execute_code', 'search_catalog', 'query_graphql'. This violates the verb_noun naming convention and forces LLMs to read full descriptions to understand tool purpose.
Descriptions are excessively long (300-500+ chars) and include implementation details ('DynamicWorkerExecutor', 'V8 isolate', 'staging helpers') irrelevant to LLM tool selection. Descriptions should be 10-200 chars, state WHAT the tool does and WHEN to use it, not HOW it works internally. Current descriptions waste tokens and bury key information.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 34 | 2026-07-28+ | v2 |
No output schema documentation. Tools return results but the response structure is not documented, LLMs cannot plan downstream calls or extract required fields. E.g., does execute_code return {result, error, logs}? Does search return {endpoints, total, next_cursor}? Undocumented outputs force LLMs to guess.
No error handling guidance. Tools execute arbitrary user code in sandboxed isolates, failures are inevitable (syntax errors, API timeouts, permission denials). Descriptions do not explain error categories, recovery steps, or what the LLM should do next. E.g., 'If code fails, try...?' is absent.
Parameter 'workspace' (optional, in execute tools) lacks description of what it controls, when to use it, or what happens if omitted. 'Optional workspace ID for shared staging' is vague, does it persist state? Affect isolation? Require setup? Undocumented dependencies cause silent misuse.