Loading ToolBench
MCP server for running Snowflake Cortex agents with support for text-to-SQL analysis, semantic search, and SQL execution
Single tool 'run_cortex_agents' has a minimal description (66 chars), adequate schema with one required parameter, but critical gaps in parameter description quality, no output schema documentation, weak error handling guidance, and missing security controls. The tool definition is visible and properly registered via @mcp.tool() decorator, but quality falls well below production baseline. Based on rubric calibration, this scores in the 'F' range (0-49) for definition quality alone.
Run the Cortex agent with the given query, streaming SSE.
Parameter 'query' lacks actionable description. Current description is empty in the source code comment; the function docstring only repeats the tool name. LLMs cannot infer expected format, length limits, or input constraints. No guidance on query language, example formats, or prerequisites.
Output schema is completely undocumented. The tool returns a dict with keys 'text', 'citations', 'sql', 'results' but no structured schema is provided to the MCP client. LLMs cannot plan what fields to expect or how to extract data downstream.
Tool description is too brief (66 chars). 'Run the Cortex agent with the given query, streaming SSE.' lacks WHAT the agent does, WHEN to use it, or what capabilities it provides. Rubric baseline: 10 - 1024 chars, optimal 50-200 for LLM selection. This example is at the p10 threshold.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 43 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 42 | - | v1 |
No error handling or recovery guidance. Tool can fail in multiple ways (network timeout, API error, malformed response, missing env vars) but returns no actionable error messages. LLM gets 'SQL execution error: <exception>' with no guidance on retry, fallback, or fix.
Credentials exposed in server initialization. SNOWFLAKE_PAT (PAT token) is loaded from environment and hardcoded into API_HEADERS. While not a direct parameter exposure, this pattern risks secrets in tool traces. Tool should validate permissions server-side, not expose auth in the request layer visible to agents.
No input validation or constraint documentation. Parameter 'query' has no type checking (though it is typed as string in schema), no length limits, no injection guards. Tool will accept any string, including malicious payloads. No guidance on SQL injection risk.
Tool name 'run_cortex_agents' uses generic verb 'run'. More specific verbs like 'query_cortex_agent' or 'search_with_cortex' would better signal to LLMs what the tool does. 'run' is ambiguous, does it execute, schedule, deploy, or invoke?
Response structure includes raw API data ('results' dict from Snowflake SQL API) without filtering or simplification. Rubric: 'APIs return verbose metadata irrelevant to chat interactions. Strip pagination internals, audit fields, and raw IDs.' This bloats context and confuses LLMs.
No tool annotation hints (readOnlyHint, destructiveHint, idempotentHint). Tool is read-only but LLMs cannot discover this from the schema. Should declare readOnlyHint=true to signal safety.