Loading ToolBench
MCP server for interacting with Google BigQuery, enabling dataset and table management, schema exploration, and SQL query execution with safety controls
This BigQuery MCP server has moderately defined tools with reasonable schemas and descriptions, but falls short of production quality. All 6 tools are explicitly registered with names, input schemas, and descriptions visible in main.py and servers/tools/bigquery.py. Naming is acceptable (verb-first convention: list_, get_, describe_, execute_, health_check). However, descriptions vary significantly in quality, some are adequate (execute_query, describe_table) while others are minimal (health_check: 'Health check MCP Server.' is only 24 chars, below the 50-char baseline for effective LLM guidance). Parameter descriptions are present across all tools but often generic ('Optional dataset ID to filter tables' lacks constraints, validation rules, or dependency hints). Output schemas are defined via Pydantic models (QueryResult, TableDetails, etc.) but not formally documented in tool docstrings where the LLM can see them. Error handling converts raw Google Cloud exceptions to ValueError but does not provide recovery guidance ('Error listing datasets: {str(e)}' offers no actionable next step). Security is reasonable (no credentials in params, read-only tools), but lacks permission declarations and audit trail guidance. The execute_query tool accepts arbitrary SQL but only validates statement type (SELECT), no mention of dry_run behavior or max_bytes_billed enforcement in the description.
Get detailed information about a specific table using INFORMATION_SCHEMA. Args: dataset_id: Dataset ID table_id: Table ID Returns: TableDetails object containing metadata about the table.
Validate a BigQuery query and optionally execute it. Args: query (str): The SQL query to execute. dry_run (bool): If True, only validate the query without executing it. Returns: QueryResult: The result of the query execution, including metadata and data.
Returns the allowed datasets configured in the environment.
Health check MCP Server.
List all datasets in the BigQuery project. If ALLOWED_DATASETS is configured, only returns those datasets.
List tables in BigQuery project, optionally filtered by dataset. Args: dataset_id: Optional dataset ID to filter tables Returns: List of tables in the specified dataset or all datasets if not specified.
health_check description is only 24 characters ('Health check MCP Server.'), below the 50-char minimum for effective LLM guidance. No context on what success looks like, when to call it, or what it returns.
No output schema documentation in tool docstrings. Pydantic models exist (QueryResult, TableDetails, ColumnDetails) but LLMs cannot see them from the tool registration. The execute_query tool description does not mention the QueryResult structure (rows, total_rows, schemas, bytes_processed, job_id, etc.), forcing LLMs to infer output shape.
Error messages do not provide recovery guidance. All tools catch exceptions and re-raise as ValueError with raw error strings. Example: 'Error listing datasets: {str(e)}' gives the LLM no hint about what to do next, retry? Check permissions? Call a different tool? Per pattern:recovery-guide, errors should state actionable next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2026-07-28+ | v2 |
| 2026-03-09 | C | 60 | - | v1 |
execute_query accepts arbitrary SQL strings but description does not document max_bytes_billed, ALLOWED_STATEMENTS constraint (SELECT only), or dry_run behavior. An LLM cannot infer that non-SELECT queries will fail or that max_bytes_billed is a server-side limit, not a parameter.
list_tables parameter dataset_id marked as optional but the description lacks dependency hints. 'Optional dataset ID to filter tables' does not explain what happens when omitted (lists all tables across allowed datasets) or what datasets are considered 'allowed' (ALLOWED_DATASETS config). An LLM cannot infer the behavior without trial and error.
No permission declarations. Tools are marked READ_ONLY but lack explicit scope annotations (e.g., 'read:bigquery', 'query:bigquery'). Pattern:scope-declaration requires each tool to declare what permissions it needs for audit and least-privilege configuration.
No pagination or result limits documented. list_datasets, list_tables, and execute_query can return large result sets. The execute_query result includes all rows without a limit parameter or documentation of max rows returned. Returning thousands of rows wastes tokens and risks context window exhaustion. Per pattern:paginated-result, tools should accept limit/offset and return total counts.
execute_query dry_run parameter defaults to false but no confirmation or warning in description. A dry_run=true call is idempotent, but dry_run=false actually executes the query and incurs costs (bytes_processed, potential side effects). Per pattern:confirmation-request, irreversible or costly operations should warn or require explicit confirmation.