Loading ToolBench
BigQuery MCP has 6 tools with consistent naming (all start with 'get_') and reasonable descriptions (avg ~120 chars). However, critical gaps undermine quality: (1) All tools have EMPTY input schemas, no parameter type definitions visible in the provided code. (2) Parameter descriptions exist but are minimal (10-30 chars), below the 72-char baseline. (3) No output schemas documented, LLMs cannot predict return structure. (4) No error handling guidance, tools fail silently. (5) All tools are READ_ONLY, reducing risk, but lack idempotent/readOnly annotations. Tool names are clear (get_datasets, get_columns) but descriptions lack WHEN/WHY context. Per-tool scores average 52; this is a D-grade server with solid naming but incomplete schemas and missing output documentation.
Use this to get all datasets having descriptions in the target region This will give a better idea of which datasets are likely to contain useful tables
Use this to get all columns in a table and their descriptions This will tell you what columns are available in a table and what they contain/how they are to be used
Use this to get a dataset with its description and other details such as when it was last updated This gives the best possible idea of whether a specific dataset is likely to contain useful tables
Use this to get all available dataset IDs This can be used to get a general idea of what collections of tables exist in the data warehouse
Use this to get the recent query history for a table This will give you a sample of recent queries that were run by analysts on a specific table
All tools have empty or missing input schemas. Tools like get_datasets and get_all_dataset_descriptions declare Input: {} with no parameter type definitions. This violates the schema requirement and prevents LLMs from understanding valid inputs.
No output schemas documented. Tools return DataFrames or Series but LLMs have no visibility into field names, types, or structure. This forces LLMs to guess what data is available and breaks tool chaining.
Parameter descriptions are too brief (10-30 chars vs 72-char baseline). E.g., 'the ID of the dataset to get details for' is minimal. Descriptions lack context on format, constraints, or when to use each tool vs alternatives.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | <=2025-11-25 | v2 |
Use this to get all tables in a dataset and their descriptions This will tell you what tables are available in a dataset and what they are likely to be useful for
No error handling guidance. Tools fail silently on invalid dataset_id or missing tables. No recovery hints (e.g., 'Call get_datasets() first to find valid IDs'). LLMs cannot self-correct.
Tool annotations missing. All tools are READ_ONLY but lack readOnlyHint in schema. No idempotentHint declared. This prevents clients from optimizing caching and retry logic.