Loading ToolBench
A Model Context Protocol server for data transformation and BI charts. Provides tools for loading data, creating visualizations, and managing BI resources with support for multiple data sources and chart types.
This server has significant gaps in definition quality. While tool names follow verb_noun conventions and descriptions are present for most tools, parameter documentation is incomplete and schemas are not fully visible in the provided source. The server implements 7 tools with varying quality: load_data, create_visualization, add_insight, create_memo, get_memo, list_memos, and delete_memo. Most parameter descriptions are present in docstrings but lack formal type constraints (enums, ranges). Error handling and output schema documentation are minimal. The memo management tools (create_memo, get_memo, list_memos, delete_memo) appear to be referenced but not fully defined in the visible source code snippet. The rubric baseline expects 100% of A-grade tools to have documented return types and parameter descriptions, this server achieves neither consistently.
Add a business insight about a visualization to the memo.
Create a new memo.
Create a visualization from the given data.
Delete a memo by ID.
Get the content of a memo by ID.
List all memo resources.
Load data from the specified source.
Output schemas are undocumented. None of the 7 tools document their return value structure or field types. The rubric baseline states 100% of A-grade tools must have documented return types. This forces LLMs to guess what fields will be present in responses, breaking tool chaining and increasing hallucination.
Parameter enums are missing. The 'format' parameter in load_data lists valid values ('auto, csv, json, sql') in the description but does not declare an enum constraint. The 'chart_type' in create_visualization similarly lists examples ('bar, line, scatter, pie, etc.') without formal enum. Free-form strings invite LLM hallucination. Per the rubric, 'When a parameter accepts one of a known set of values, declare it as an enum.'
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 41 | - | v1 |
Parameter range and validation rules are undocumented. No tool specifies min/max for numeric params (e.g., if page_size or limit apply to list_memos, what are the bounds?). The 'options' object in create_visualization is typed as object but contains no schema for valid nested fields. Per the rubric baseline, 194 chars is the average tool description, these are shorter and lack actionable constraints.
Destructive tools lack confirmation/dry-run pattern. delete_memo is marked as DESTRUCTIVE but the visible code shows no confirmation step, dry-run mode, or compensation tool (e.g., restore_memo). The rubric pattern states: 'Irreversible operations should support a dry-run or confirmation step. Agents make mistakes, a confirm_before_execute pattern prevents catastrophic errors.'
Error handling lacks recovery guidance. The visible code returns generic dictionaries like {"success": True, "message": "..."} with no error classification or next-step guidance. The rubric states: 'Error responses must tell the LLM what to do next: "User not found. Try search_users() with a partial name." A raw error code gives the agent nothing to act on.'
Memo tools (create_memo, get_memo, list_memos, delete_memo) are referenced but not fully defined in the source snippet. The code shows 'from .resources.memo import MemoResourceManager' and 'self.register_memo_tools()' but the actual tool definitions are cut off. Per the hard scoring rule: 'If you cannot see the actual tool definition in the source (only inferred), cap that tool's overall at 50.' All four memo tools are capped accordingly.
Tool descriptions lack LLM-optimized guidance. Per the rubric, descriptions should be 10 - 1024 characters and answer WHAT, WHEN, and WHY. 'create_visualization' is 87 chars, acceptable length but missing dependency hints like 'First call load_data() to inspect available columns.' Tool discovery is implicit, forcing LLMs to reason about call order.
Pagination support is unclear. list_memos takes no visible parameters (empty input {}). If the memo database is large, returning all memos will blow the context window. Per the rubric: 'Tools returning lists should accept page/offset and limit parameters and return a total count or next_cursor.'
No tool composition chain documented. The workflow 'load_data → create_visualization → add_insight' is implicit. No tool description hints that add_insight requires a visualization_uri from create_visualization. Per the rubric: 'Ensure tool A's output contains the IDs and references tool B needs.' Unclear whether create_visualization.return.visualization_uri matches add_insight.input.visualization_uri.