Loading ToolBench
MCP server for querying Power BI XMLA endpoints and retrieving table metadata, columns, and executing SQL queries
Server provides 3 tools with reasonable naming and descriptions, but lacks comprehensive parameter documentation, output schema specifications, and error handling guidance. All tools follow verb_noun naming convention (get_*, run_*) which is good. However, parameter descriptions are minimal (single-phrase only), output format is undocumented (only mentioned as 'CSV format'), and error responses appear to throw raw RuntimeExceptions without recovery guidance. The server meets basic definition standards but falls short of production-grade quality on output documentation and error handling.
Retrieves a list of fields, dimensions, or measures (as columns) for an object, entity or collection (table). Use the `{prefix}_get_tables` tool to get a list of available tables. The output of the tool will be returned in CSV format, with the first line containing column headers.
Retrieves a list of objects, entities, collections, etc. (as tables) available in the data source. Use the `{prefix}_get_columns` tool to list available columns on a table. Both `catalog` and `schema` are optional parameters. The output of the tool will be returned in CSV format, with the first line containing column headers.
Execute a SQL SELECT statement.
Output schemas are not documented. Tools return CSV format but no schema for columns, data types, or field structure is specified. LLMs cannot understand result structure to plan downstream operations.
Parameter descriptions are single-phrase only ('The catalog name', 'The table name'). Missing context on format, constraints, whether optional/required, and what happens when values are invalid. Rubric baseline requires 72 chars avg per parameter description.
pbi_run_query description is severely truncated (52 chars). Does not explain when to use this vs discovery tools, SQL dialect limitations beyond 'mostly SQL-92', or what happens on syntax errors. Does not meet 10-1024 char rubric minimum for actionable descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 0 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 46 | - | v1 |
Error handling throws raw RuntimeException('ERROR: ' + ex.getMessage()). No guidance on recovery, no categorization of error type (retryable vs user-fixable vs fatal), no actionable error messages. Rubric requires: 'Error responses must tell the LLM what to do next.'
pbi_get_columns 'table' parameter is marked required in code (.required('table')) but no parameter-level description exists to explain what happens if omitted. Code comment alone does not surface to LLM.
pbi_run_query accepts a 'sql' parameter with complex constraints (SQL-92, quoted identifiers, allowed clauses) but constraint list is buried in description as prose. Should be stated as clear requirements with examples: 'Valid clauses: FROM, INNER JOIN, LEFT JOIN, GROUP BY, ORDER BY, LIMIT/OFFSET; identifiers quoted with ""'.
No pagination support documented. If result sets are large, tool returns all rows as CSV, risking context window exhaustion. Rubric requires: tools returning lists should accept limit/offset and return total count. No evidence of pagination in GetTablesTool or RunQueryTool.