Loading ToolBench
MCP server providing SQL query execution tools for BigQuery tables containing customer, product, sales, feedback, and threat intelligence data
This MCP server exposes 8 BigQuery SQL-execution tools with significant quality gaps. All tools follow a consistent pattern: accept arbitrary SQL strings and return raw BigQuery results. While descriptions are present and detailed, they fail critical LLM-optimization requirements. Tool names violate verb-first conventions (e.g., 'Bigquery_Customer', 'Oracle_CustomerFeedback' are nouns, not action verbs). Descriptions are excessively long (500+ chars) and include concrete example SQL queries that LLMs tend to reuse literally. Input parameters ('sql' only) lack enum constraints, format specifications, or length limits, inviting SQL injection and hallucinated queries. Output schemas document fields at a high level but don't constrain response structure for pagination, error clarity, or chaining. Error handling is minimal, catches and returns raw exception strings, providing no recovery guidance. Security is not addressed: accepting arbitrary SQL is a fundamental design flaw for an agent tool; there is no input sanitization, rate limiting, or audit trail. The 'sql' parameter has no description, violating the parameter annotation requirement. Tools are undifferentiated, each is mechanically identical, differing only in which table they query. This defeats tool composition: an LLM cannot reason about which tool to use based on intent.
Execute a SQL query on the BigQuery `daily_market_indices_with_news` table containing daily financial market indices and related news headlines. **Schema:** `genai-poc-424806.MCP_demo.daily_market_indices_with_news` - Date (DATE) - AAPL (FLOAT64) — Apple Inc. closing price - NASDAQ (FLOAT64) — NASDAQ Composite index closing value - NYA (FLOAT64) — NYSE Composite index closing value - SP500 (FLOAT64) — S&P 500 index closing value - DJI (FLOAT64) — Dow Jones Industrial Average closing value - Headline (STRING) — Key market-related news headline for the day Use this tool to: - Retrieve historical or recent market performance data - Correlate market movements with relevant news headlines - Filter by specific dates or date ranges - Compare performance across indices on given dates **Example:** SELECT Date, SP500, Headline FROM `genai-poc-424806.MCP_demo.daily_market_indices_with_news` WHERE Date >= '2025-01-01'
Execute a SQL query on the BigQuery `Customer` table containing customer profile information. **Schema:** `genai-poc-424806.MCP_demo.Customer` - CustomerID (INT64, PRIMARY KEY) - FirstName (STRING) - LastName (STRING) - Email (STRING) - Phone (STRING) - Address (STRING) - JoinDate (DATE) Use this tool to: - Retrieve customer demographic and contact information - Filter customers by join date, name, or location - Join customer data with sales, feedback, or call logs **Example:** SELECT * FROM `genai-poc-424806.MCP_demo.Customer` WHERE JoinDate >= '2024-01-01'
Execute a SQL query on the BigQuery `DMV_Customer_Feedback` table containing dmv customer feedback with feedback score range. **Schema:** `genai-poc-424806.MCP_demo.DMV_Customer_Feedback` - Response Date (Timestamp) - Feedback (String) — DMV customer feedback text - Score (String) — DMV customer feedback score range (e.g., "1-2", "3-5"). The Score range (1-2) is considered negative feedback, while (3-5) is considered positive feedback. Use this tool to: - Retrieve DMV customer feedback text with score range - Analyze customer satisfaction trends over time - Filter feedback by specific score ranges or dates **Example:** SELECT * FROM `genai-poc-424806.MCP_demo.DMV_Customer_Feedback` where Score='(3-5)' and `Response Date`>'2025-07-05'
All tool names violate verb-first naming convention. Names like 'Bigquery_Customer', 'Oracle_CustomerFeedback', 'Cloud_SQL_Product' are nouns describing resources, not action verbs. LLMs use tool names as the primary signal for intent before reading descriptions. The server should use names like 'query_customer_table', 'get_product_info', 'search_customer_feedback'.
The 'sql' parameter has NO description. It is the only input parameter to all 8 tools, yet it lacks guidance on format, constraints, or valid syntax. The rubric requires every parameter to have a non-empty description explaining what it controls. Without it, LLMs cannot reason about parameter values or constraints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 39 | 2026-07-28+ | v2 |
Execute a SQL query on the BigQuery `Product` table containing product catalog information. **Schema:** `genai-poc-424806.MCP_demo.Product` - ProductID (INT64, PRIMARY KEY) - ProductName (STRING) - Category (STRING) - Price (FLOAT64) - StockQuantity (INT64) - AddedDate (DATE) Use this tool to: - Retrieve product details, categories, and pricing - Filter products by category, stock levels, or date added - Support product catalog analytics and inventory management **Example:** SELECT * FROM `genai-poc-424806.MCP_demo.Product` WHERE Category = 'Electronics'
Execute a SQL query on the BigQuery `CustomerFeedback` table containing customer reviews and comments. **Schema:** `genai-poc-424806.MCP_demo.CustomerFeedback` - FeedbackID (INT64, PRIMARY KEY) - CustomerID (INT64) - ProductID (INT64) - SalesID (INT64) - FeedbackDate (DATE) - Feedback (STRING) Use this tool to: - Retrieve customer feedback and satisfaction data - Filter reviews by date, product, or sentiment keywords - Link feedback to sales transactions for quality analysis **Example:** SELECT * FROM `genai-poc-424806.MCP_demo.CustomerFeedback` WHERE ProductID = 101
Execute a SQL query on the BigQuery `Sales` table containing transaction records. **Schema:** `genai-poc-424806.MCP_demo.Sales` - SalesID (INT64, PRIMARY KEY) - CustomerID (INT64) - ProductID (INT64) - Quantity (INT64) - TotalAmount (FLOAT64) - SaleDate (DATE) Use this tool to: - Analyze total sales amounts and quantities - Filter transactions by date, customer, or product - Aggregate sales metrics for reporting and dashboards **Example:** SELECT CustomerID, SUM(TotalAmount) FROM `genai-poc-424806.MCP_demo.Sales` GROUP BY CustomerID
Query the BigQuery `threat_iocs` table containing threat intelligence indicators of compromise (IOCs). **Schema:** `genai-poc-424806.MCP_demo.threat_iocs` - ioc_type (STRING) — Type of IOC (IP, URL, domain, hash) - value (STRING) — Actual IOC value - threat_actor (STRING) — Associated threat actor - source (STRING) — Source of the IOC - confidence (STRING) — Confidence level - first_seen (TIMESTAMP) - last_seen (TIMESTAMP) - ttps (STRING) — Tactics, Techniques, and Procedures Use this tool to: - Investigate IOCs and their context - Filter by time, threat actor, or confidence - Link IOCs with alerting, threat feeds, or TTPs **Example:** SELECT * FROM `genai-poc-424806.MCP_demo.threat_iocs` WHERE threat_actor = 'APT29'
Execute a SQL query on the BigQuery `CustomerCallLog` table containing customer service call history. **Schema:** `genai-poc-424806.MCP_demo.CustomerCallLog` - CallID (INT64, PRIMARY KEY) - CustomerID (INT64) - CallDate (DATE) - CallReason (STRING) - CallDurationMinutes (INT64) - CallNotes (STRING) Use this tool to: - Retrieve and analyze customer service interactions - Filter call logs by reason, duration, or date - Track customer engagement and support history **Example:** SELECT * FROM `genai-poc-424806.MCP_demo.CustomerCallLog` WHERE CallReason = 'Product Inquiry'
Descriptions embed concrete SQL example queries (e.g., 'SELECT * FROM ... WHERE JoinDate >= 2024-01-01'). The rubric explicitly warns: 'Do not put example values in the description. LLMs tend to reuse example values literally rather than adapting to the actual context.' All 8 tools violate this. Descriptions should explain WHEN to use the tool, not show copy-paste examples.
Descriptions are excessively long (500 - 700+ characters). The rubric specifies 10 - 1024 chars, but recommends keeping them 'between 50 - 200 chars for LLM optimization.' Lengthy descriptions waste tokens, bury key intent signals, and increase hallucination likelihood. Descriptions should state WHAT the tool does, WHEN to use it, and what it returns, not enumerate schema fields and include SQL examples.
No input validation or constraints on the 'sql' parameter. The tool accepts arbitrary SQL strings, allowing an LLM (via prompt injection) to craft malicious queries: DROP TABLE, DELETE FROM, exfiltrate data, cause denial of service. This is a fundamental security failure. Tools should validate input against expected patterns, reject dangerous operations, and log access. See pattern:tool-gateway.
Error handling is minimal and unhelpful. All 8 tools catch exceptions and return {"error": str(e)}, which provides the raw exception message (stack trace, API details). The rubric requires errors to guide recovery: 'Error responses must tell the LLM what to do next.' A raw error string like 'Table not found' gives the agent no actionable path forward. Errors should be categorized (retryable vs. unrecoverable), include context (available tables, valid query syntax), and suggest recovery steps.
Output schema is documented in natural language within the description but not formally specified. The MCP spec requires structured output schemas. The current return format {"table": <string>, "row_count": <int>, "rows": <list>} is not validated or constrained. Rows are arbitrary dictionaries with no type information. Pagination is absent, the tool returns all results, which can exhaust context windows and cause timeouts.
All 8 tools are mechanically identical, they differ only in which table they query. This defeats tool composition and LLM intent reasoning. A single 'query_bigquery' tool with a 'table' parameter would be clearer and more maintainable. Alternatively, each tool should be named to reflect its domain action: 'get_customers', 'list_products', 'analyze_sales', 'search_feedback', not 'Bigquery_Customer' and 'Oracle_CustomerFeedback'.
No audit trail, logging, or permission gates. The server does not log which user called which tool, with what SQL, at what time, or with what result. Agents running arbitrary SQL against production data is a compliance and security risk. Implement per-tool audit logging, user attribution, and query-level permissions. See pattern:audit-trail and pattern:permission-gate.
No rate limiting or timeout enforcement. A malfunctioning agent can issue unlimited BigQuery calls, each potentially scanning terabytes of data and incurring massive cloud costs. Tools should enforce per-request timeouts, per-agent rate limits, and result size caps. See pattern:rate-limit (if available) or implement directly.