Loading ToolBench
Full-featured MCP server for Azure Synapse Analytics — SQL, Pipelines, Spark, and Notebooks
Azure Synapse MCP has 12 tools with complete input schemas and descriptions, but quality is inconsistent. Naming follows verb_noun convention well (list_*, run_*, get_*, describe_*). Descriptions are present but brief (avg ~60 chars, below the 194-char baseline). Critical gaps: no output schemas documented, no error handling guidance, no parameter descriptions for most inputs, and destructive tools (run_statement, run_pipeline, submit_spark_job) lack confirmation/dry-run patterns. Parameters like 'sql' and 'file' lack format/constraint documentation. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels. Composition is reasonable, tools are single-purpose, but chaining support is weak (e.g., run_pipeline returns run_id but no pipeline metadata).
Get column names, types, and nullability for a table.
Check the status of a pipeline run by run ID.
Get row count and size statistics for a table.
List all databases in the Azure Synapse workspace.
List all notebooks in the Synapse workspace.
List all pipelines in the Synapse workspace.
No output schemas documented. LLMs cannot plan downstream calls or extract required fields (e.g., what fields does run_pipeline return? Is run_id always present?). Baseline: 100% of A+ tools document return types.
Destructive tools (run_statement, run_pipeline, submit_spark_job) lack confirmation/dry-run patterns. Agents can execute DELETE or DROP without safeguards. No error guidance for recovery.
Parameter descriptions missing or minimal. 'sql' parameter lacks format/constraint guidance (SELECT only? Max length?). 'file' in submit_spark_job lacks ABFSS path format validation. 'executor_size' has no enum constraint (Small|Medium|Large?).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | <=2025-11-25 | v2 |
List recent Spark batch jobs on the configured Spark pool.
List all tables (and views) in a Synapse database.
Trigger a pipeline run. Returns the run ID.
Execute a SQL SELECT query and return results as JSON.
Execute a DML or DDL statement (INSERT, UPDATE, DELETE, CREATE, DROP). Returns rowcount.
Submit a PySpark script as a Spark batch job.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint). Tools are marked with Risk levels in metadata but not exposed via MCP tool annotations. LLMs cannot infer safety properties.
No error handling guidance. Errors (e.g., 'Database not found', 'SQL syntax error') lack recovery hints. LLMs cannot self-correct or know whether to retry, ask user, or abort.