Loading ToolBench
Enterprise MCP Gateway — reference implementation (gateway on FastMCP substrate)
Four tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. Tool names follow verb_noun convention (list, update, get, post). Descriptions exist but are terse (10-60 chars) and lack context on when to use each tool or what they return. Input schemas present with type info, but parameter descriptions are minimal. No output schemas documented. Error handling and recovery guidance absent. Risk annotations (READ_ONLY, WRITE) are present but not formalized in tool definitions.
List CRM customers (read-only).
Update a CRM customer's tier. Elevated — see the gateway tool registry.
Get an account balance (read-only).
Post a ledger entry. Elevated — see the gateway tool registry.
Output schemas not documented. LLMs cannot infer what fields each tool returns, forcing them to guess downstream field names and risking broken tool chains.
Descriptions are too brief (10-60 chars). They lack context on WHEN to use each tool, WHAT it returns, and any prerequisites. Descriptions should be 50-200 chars and answer: what does it do, when should the LLM call it, what does it return?
No pagination support visible in crm_list_customers or ledger_get_balance. Large result sets will blow context windows. Add limit and offset/cursor parameters and return total_count.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | <=2025-11-25 | v2 |
No error handling or recovery guidance. Tools lack descriptions of failure modes, retryability, or what the LLM should do if a call fails (e.g., 'customer not found, try search_customers first').
Risk annotations (READ_ONLY, WRITE) are metadata but not formalized as tool annotations in the schema. Modern MCP supports readOnlyHint and destructiveHint, use these to signal intent to clients.