Loading ToolBench
Safe, read-only MCP server for Microsoft Fabric - 25 tools for data engineers
The server has 24 well-named, READ_ONLY tools with consistent verb_noun naming (list_*, get_*). All tools have descriptions and input schemas are visible in the codebase. However, descriptions are uniformly brief (15-50 chars), falling below the 50-200 char ideal for LLM optimization. Parameter descriptions are generic ('Workspace name or ID', 'Lakehouse name or ID') and lack actionable guidance on format, constraints, or error recovery. Output schemas are not documented in tool definitions, the LLM cannot predict response structure. Tool composition is strong (each tool does one thing, names are unambiguous), but descriptions need depth to guide LLM selection and error handling is absent (no error recovery guidance or user-fixable hints).
List all capacities the user has access to
Get detailed information about a connection
Get detailed information about a specific item
Get details about a specific job instance
Get detailed information about a lakehouse
Get details about a specific schedule
Get storage information for a lakehouse
Descriptions are uniformly brief (15-50 chars) and lack LLM-optimized guidance. Examples: 'List all connections the user has access to' (44 chars), 'List all Spark environments in a workspace' (41 chars). The rubric baseline for optimal descriptions is 50-200 chars with explicit WHAT/WHEN/WHY guidance. Current descriptions lack context on when to use each tool vs. similar alternatives (e.g., when to call list_items vs. list_lakehouses).
Parameter descriptions are generic and lack actionable format/constraint guidance. All workspace and lakehouse parameters use 'Workspace name or ID' or 'Lakehouse name or ID' without clarifying: (1) which format is preferred, (2) whether both are valid, (3) examples of valid inputs, or (4) what happens if both are provided. The rubric requires 'state the expected format, range, and allowed values directly in the parameter description.' Current descriptions violate this principle for ~40 parameters across the tool set.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Get column statistics for a table
Get sample data from a table
Get the schema of a specific table
Get row count and other statistics for a table
Get detailed information about a workspace
List all connections the user has access to
List all Spark environments in a workspace
List all items in a workspace
List job instances for a workspace or item
List all lakehouses in a workspace
List all schedules for a workspace or item
List all shortcuts in a lakehouse
List all partitions for a table
List all tables in a lakehouse
List all available workspaces
Query a table with SQL SELECT statement (read-only)
Search for items by name in a workspace
Output schemas are not documented in tool definitions. The codebase contains API client methods (get_workspaces, get_lakehouses, get_tables, etc.) that return List[Dict], but there is no schema documentation describing the structure of returned objects, field types, or which fields the LLM should expect. This forces the LLM to reason about response structure without guidance, increasing hallucination risk. The rubric requires 'Document the output schema. LLMs need to know what fields to expect so they can plan downstream tool calls.' None of the 24 tools include output schema documentation.
No error handling or recovery guidance. Tools have no visible error handling code in tool definitions, and responses do not guide LLMs on recovery actions. For example, query_table accepts a 'where_clause' parameter but there is no schema validation or error message like 'Invalid WHERE clause syntax. Use: WHERE column_name = value AND other_column > threshold'. Missing error guidance violates the pattern 'Error responses must tell the LLM what to do next.' Current code logs errors to stderr but does not return actionable error messages to the agent.
Pagination is implemented server-side (FabricApiClient.paginated_request) but not exposed to the LLM through tool parameters. The rubric requires 'Tools returning lists should accept page/offset and limit parameters and return a total count or next_cursor.' List tools (list_workspaces, list_lakehouses, list_tables, list_items, list_shortcuts, etc.) use automatic pagination internally, but LLM cannot control page size or fetch specific pages. This may return variable amounts of data and prevents the LLM from controlling result volume, risking context window exhaustion. The query_table tool has a 'limit' parameter (good), but list_* tools do not.
Parameters accept flexible identifiers (name or ID) but lack disambiguation in schema. Tools document 'Workspace name or ID' but the JSON Schema input does not use enums, patterns, or typed sub-fields to distinguish the two. LLMs must infer whether to pass a UUID, a display name, or both. The rubric recommends 'Provide separate parameters (channel_id, channel_name) rather than a single overloaded parameter.' Alternatively, parameter description should explicitly state 'UUID format (36 chars) or workspace display name (string, case-insensitive).'
Missing tool-to-tool composition guidance. Multiple list_* tools exist (list_items, list_lakehouses, list_tables, list_shortcuts, list_job_instances, list_schedules) that retrieve metadata before calling detail tools. Descriptions do not hint at the discovery pattern, e.g., 'Call list_workspaces() first to get available workspace IDs, then pass to list_lakehouses().' This forces LLMs to discover these call sequences through trial, wasting tokens and latency.
query_table tool description is 'Query a table with SQL SELECT statement (read-only)' which is good for stating the scope, but lacks guidance on supported SQL features, WHERE clause syntax, or examples. The 'where_clause' parameter accepts free-form SQL but has no validation hint. LLMs may attempt unsupported SQL syntax (JOINs, subqueries, aggregations) leading to errors. Description should clarify 'Supports simple WHERE clauses. Examples: name='John' OR age > 25. Does not support JOINs or subqueries.' This matches the rubric: 'Write descriptions as if prompt-engineering. State WHAT the tool does, WHEN to use it, and any prerequisites.'