Loading ToolBench
MCP server for querying ORION-DBs (open research information databases) via BigQuery. Provides schema browsing, cost estimation, query execution, and result analysis tools for research data exploration.
Strong workflow-oriented design with clear step-by-step progression (list → explore → schema → estimate → execute → analyze). All 10 tools have detailed descriptions (avg 250+ chars, well above 194-char baseline). Tool names follow verb_noun pattern (orion_list_*, orion_get_*, orion_run_*, orion_result_*). Input schemas present for 7/10 tools with proper type_string() declarations. Critical gap: 3 tools (orion_health_check, orion_list_datasets, orion_run_bq_query) lack visible input schema definitions in source, though orion_run_bq_query is registered in server.R. Descriptions are LLM-optimized with explicit WHEN/WHY guidance (e.g., 'Use when the user asks...', 'PREREQUISITE:', 'CRITICAL:'). Error handling guidance present in descriptions but not formalized as structured error responses. No tool annotations (readOnlyHint, destructiveHint) despite all tools being READ_ONLY.
STEP 4 OF QUERY WORKFLOW: Perform a BigQuery dry run to validate SQL syntax and estimate scan cost. This does NOT execute the query — it only returns estimated bytes processed. Dry runs are completely free and must ALWAYS be called before orion_run_bq_query. Takes exactly one argument — query — which must be the complete, fully-qualified SQL string. Cost calculation: $6.25 per TB scanned (1 TB = 1,000 GB; the approximate BigQuery on-demand rate). After receiving the estimate, STOP and present it following the returned message exactly: show the full SQL in a fenced code block, explain in plain language what it does, state the GB scanned and cost, then ask 'Shall I run this query?'. Do NOT call orion_run_bq_query until the user explicitly confirms they want to proceed. CRITICAL: the SQL passed to orion_run_bq_query MUST be byte-for-byte identical to the SQL passed here. Do NOT reformat, rewrite, or improve the SQL after the dry run. If the user asks to change the query in any way, call orion_estimate_query_cost again with the new SQL before presenting a cost estimate or running it. Never reuse a previous cost estimate for modified SQL. If the estimate exceeds 100 GB, highlight this prominently and strongly recommend the user confirm.
ANALYSIS TOOL (after STEP 5): Export a stored query result to a file (CSV for flat data, JSON Lines for nested data). Operates on the FULL result held in the R session, not the truncated preview. Use when you need to save results for further analysis or sharing. Costs nothing — no BigQuery access involved.
STEP 3 OF QUERY WORKFLOW: Get the full BigQuery schema for a specific table. Call this after orion_list_tables to understand column names and types before writing SQL. Also useful when comparing table structures across datasets. Schema reading rules: - REPEATED fields must be flattened with UNNEST() in queries. - RECORD fields are accessed via dot notation (e.g. open_access.oa_status). - Identify the primary identifier column (e.g. doi, id) — you will need it for COUNT DISTINCT and joins.
Three tools (orion_health_check, orion_list_datasets, orion_run_bq_query) lack visible input schema definitions in source code.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all 10 tools being READ_ONLY operations. Annotations enable agents to reason about safety and retry semantics.
Error handling is documented in descriptions (e.g., 'PREREQUISITE:', 'CRITICAL:') but not formalized as structured error responses with recovery guidance. Agents cannot programmatically distinguish retryable vs. fatal errors.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
SETUP CHECK: Verify the orion-mcp installation end to end — schema metadata, Google credentials, billing project, BigQuery connectivity (via a free dry run), and the export folder mount. Completely free and safe to run at any time. Use when the user asks whether the tool is working, after installation, or when a query fails in a way that suggests a setup problem (missing credentials, billing project, network). Walk the user through each failed check's fix step by step.
STEP 1 OF QUERY WORKFLOW: List all ORION-DBs datasets available on BigQuery from various providers of open research information. Does NOT return schemas — call orion_list_tables next to explore a specific dataset. Use this as the entry point whenever the user asks about available data or wants to run a query. IMPORTANT: Use the dataset descriptions as-is — do NOT infer recency from dates embedded in dataset names. 'instant' means the most recent snapshot. For more info see <https://orion-dbs.community/>
STEP 2 OF QUERY WORKFLOW: List all tables in a specific project/dataset with descriptions. Call this after orion_list_datasets to identify which table to query. Use the table descriptions to pick the right table before fetching its full schema.
ANALYSIS TOOL (after STEP 5): Frequency counts over one or more columns of a stored query result, sorted by count (top 100 groups). Operates on the FULL result held in the R session, not the truncated preview. Use for questions like 'how does this break down by year / institution / type?' without re-querying BigQuery. Costs nothing — no BigQuery access involved.
ANALYSIS TOOL (after STEP 5): Return selected rows and columns of a stored query result, optionally filtered. Operates on the FULL result held in the R session, not the truncated preview. Use to drill down into specific subsets without re-querying BigQuery. Costs nothing — no BigQuery access involved.
ANALYSIS TOOL (after STEP 5): Per-column summary of a stored query result (type, missing values, distinct count, min/mean/max or example values). Operates on the FULL result held in the R session, not the truncated preview. Use this first when a query result was truncated, to understand the data before slicing or counting. Costs nothing — no BigQuery access involved.
STEP 5 OF QUERY WORKFLOW: Execute a BigQuery SQL query, return results, and summarise them in plain language. PREREQUISITE: orion_estimate_query_cost MUST have been called for this exact SQL AND the user must have explicitly confirmed they want to proceed after seeing the cost. Never skip the cost confirmation step, even for small queries. Takes exactly one argument — query — which must be the complete, fully-qualified SQL string. Every result is stored in the R session under a stable name (q1, q2, ...) and only a size-capped preview is returned inline. If the response says the preview was TRUNCATED, do not reason from the partial rows — use orion_result_summary, orion_result_count, or orion_result_slice on the stored result, or orion_export_result to save it to a file. All of these are free: they use the data already in the R session and never re-query BigQuery. SQL rules: - Never use SELECT * — name only the columns needed to minimise bytes scanned. - Never use COUNT(*) — always count distinct over a unique identifier (e.g. COUNT(DISTINCT doi)). - Always lowercase identifiers before joining across collections: LOWER(doi), LOWER(orcid), LOWER(issn). - DOI fields may be stored as bare DOIs ('10.1234/foo') or as URLs ('https://doi.org/10.1234/foo') depending on the dataset. - Always normalise DOIs before joining using REGEXP_REPLACE (not REGEXP_EXTRACT) to avoid introducing NULLs: LOWER(REGEXP_REPLACE(doi, r'^https?://doi\.org/', '')) — this safely strips the prefix if present and leaves bare DOIs unchanged. - Use ROR IDs when searching for an institution first
orion_result_slice has 7 optional parameters (columns, filter_column, filter_op, filter_value, n) with interdependencies not documented. E.g., filter_op requires filter_column and filter_value; filter_column requires filter_op. Undocumented dependencies cause silent misuse.
Output schemas not documented for any tool. LLMs cannot plan downstream calls or extract fields without knowing what each tool returns. E.g., orion_list_datasets returns dataset list but structure is not specified.