Loading ToolBench
A Model Context Protocol server for data platform operations, integrating with DataHub, Trino, S3, and notification systems
Single tool 'notify' has a moderately detailed description (~280 chars) and comprehensive input schema with 7 parameters. However, the schema lacks proper JSON Schema type declarations visible in the source, parameter descriptions are present but minimal (10-30 chars each), and there is no documented output schema. The tool description explains WHAT it does and mentions three action types (list/send/publish), but lacks guidance on WHEN to use it or any prerequisite steps. Error handling, validation, and recovery guidance are absent. This server has foundational definition elements but falls well short of production-grade tool design.
Send a message, or publish a saved asset, to a notification channel an administrator has configured (Slack, Mattermost, an incoming webhook, or an email list). Call action=list first: channel names are not guessable, and a channel this session cannot reach is not listed. action=send posts a title and a markdown body with an optional link. action=publish posts a portal asset -- its title, its content, and a link back to it. Delivery is queued, so success means the message was accepted for delivery, not that it has already appeared. This tool does not create channels.
Output schema completely undocumented. Tool returns an unspecified response structure, forcing LLMs to guess what fields are available for downstream chaining or error detection.
Parameter descriptions are minimal (10-30 chars). Example: 'channel' is described as 'the channel name, as action=list reports it', this is contextual but lacks actionable format guidance. 'body' says only 'the message body as markdown' without mentioning length limits, required formatting, or examples. Parameter descriptions should average 50-100 chars with explicit constraints.
No input validation or error recovery guidance. The tool accepts 'action' as a free-form string with only implicit documentation of valid values (list, send, publish). No error messages documented for invalid actions, missing required fields, or delivery failures. Agents cannot self-correct on failure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | D | 52 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Parameter 'action' should be an enum constraint, not a free-form string. Free-form 'action' invites hallucinated values. Declare as enum: ['list', 'send', 'publish'].
Conditional parameter dependencies are undocumented. 'title' and 'body' are required for action=send but optional for action=list. 'asset' is required for action=publish. LLMs will pass incomplete parameter sets, causing silent failures or confusing error responses.
No dry-run or confirmation pattern for destructive/irreversible operations. The tool description notes 'Delivery is queued, so success means the message was accepted for delivery', implying an irreversible side effect, but offers no way for agents to preview, confirm, or safely test before committing.
Tool name 'notify' is a generic verb that does not distinguish between three distinct operations (list channels, send message, publish asset). A clearer naming scheme like 'send_notification' with separate tools for discovery and publishing would better match the single-responsibility principle and improve LLM intent inference.