Loading ToolBench
A Model Context Protocol server that provides tools for querying Office 365 data through a JDBC interface, enabling SQL-based access to tables, columns, and data retrieval.
Office 365 MCP Server has three well-named, read-only tools with documented schemas and descriptions. However, there are significant gaps in parameter descriptions, output schema documentation, and error handling guidance. Tool names follow verb_noun convention (get_tables, get_columns, run_query) which is strong. Descriptions are present but inconsistent in detail, get_tables and get_columns include helpful cross-references to related tools, but run_query lacks guidance on SQL dialect constraints and available functions. Parameter descriptions exist but are minimal (e.g., 'The catalog name' without cardinality, format, or dependency hints). Output is described as 'CSV format' but no structured schema is documented for downstream tool chaining. Error handling is missing entirely, users have no recovery guidance if a query fails. The code shows proper tool registration via ITool interface and JsonSchemaBuilder, indicating a mature implementation approach, but the definitions themselves lack the depth production agents require.
Retrieves a list of fields, dimensions, or measures (as columns) for an object, entity or collection (table). Use the `office365_get_tables` tool to get a list of available tables. The output of the tool will be returned in CSV format, with the first line containing column headers.
Retrieves a list of objects, entities, collections, etc. (as tables) available in the data source. Use the `office365_get_columns` tool to list available columns on a table. Both `catalog` and `schema` are optional parameters. The output of the tool will be returned in CSV format, with the first line containing column headers.
Execute a SQL SELECT statement.
No structured output schema documented for any tool. All three tools return CSV format, but LLM has no machine-readable definition of fields, types, or whether results are paginated. This prevents confident downstream chaining and increases hallucination risk.
Error handling entirely absent. No guidance on what to do if a query fails, table not found, connection timeout, or SQL syntax error occurs. LLM has no recovery path.
Parameter descriptions are minimal and lack format/constraint details. 'The catalog name' tells LLM nothing about valid formats, whether it's case-sensitive, or what happens if omitted. run_query's SQL parameter lacks max length, validation rules, or examples of valid/invalid syntax.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 33 | - | v1 |
run_query description is only 28 characters ('Execute a SQL SELECT statement'). Hard cap rule applies: description score capped at 40. Critical constraints (SQL-92 dialect, quoted identifiers, allowed clauses) are buried in inline text rather than structured as parameter metadata or examples.
No pagination or result-size limits documented. If get_tables or get_columns return hundreds of tables/columns, response token cost explodes. Description should state 'Results are capped at X; use catalog/schema filters to narrow results.'
Required vs optional parameters not clearly marked. JsonSchemaBuilder code shows only .required('table') for get_columns; catalog and schema are optional but descriptions do not state this explicitly. LLM may pass them unnecessarily or omit them when needed.