Loading ToolBench
Template-driven deployment system for Model Context Protocol servers with Docker, Kubernetes, or Mock backends. Provides zero-configuration deployment and template discovery.
This is a template/example MCP server repo with 16 tools split across demo and Zendesk templates. Tools have descriptions and basic schemas, but quality is inconsistent. Demo tools (say_hello, get_server_info, echo_message, demonstrate_overrides) are instructional examples with adequate descriptions. Zendesk tools (create_ticket, get_ticket, update_ticket, etc.) have descriptions and type information visible in the evaluation data, but critical gaps exist: (1) No explicit error handling guidance visible in source, tools lack recovery hints or error classification. (2) Output schemas are undocumented, no visible specification of what these tools return, field names, or structure. (3) Parameter descriptions are minimal, most params lack detail about format, constraints, or dependencies. (4) No tool annotations (readOnlyHint, destructiveHint, idempotentHint) visible. (5) Security patterns absent, no documented permission gates or scope declarations. (6) Composition issues: create_ticket and create_user both make mutations but lack idempotency or dry-run support. The Zendesk subset appears to be a template/example showing API integration patterns rather than production-grade implementations. Average per-tool score ~52, placing this in the 'D' (Poor) range, significant gaps in descriptions, schemas, and error handling.
Add a public or private comment to an existing Zendesk ticket.
Create a new Zendesk support ticket with subject, description, priority, and other details.
Create a new user in Zendesk with email and optional profile information.
Demonstrate the two configuration patterns: standard config from template.json config_schema and template data overrides via double underscore notation.
Echo back a message with server identification. Demonstrates template data override for tool behavior via custom echo prefix.
Retrieve a specific Zendesk knowledge base article by ID.
Output schemas completely undocumented. No visible specification of return types, field names, structure, or pagination for any tool. Tools cannot be reliably chained or reasoned about by LLMs without understanding their output.
No error handling guidance or recovery hints. Error responses will not tell LLMs what to do next (retry, ask user, give up). No error classification (retryable vs user-fixable vs fatal).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | D | 59 | <=2025-11-25 | v2 |
| 2026-03-09 | C | 60 | - | v1 |
Get information about the demo server. Returns server metadata including name, version, description, standard configuration values, registered tools, and custom fields.
Retrieve details of a specific Zendesk ticket by ID.
Retrieve ticket metrics and analytics for a specified time period.
Retrieve details of a specific Zendesk user by ID or email.
List all organizations in Zendesk with pagination support.
Generate a personalized greeting message. Uses standard config from config_schema (hello_from parameter) and supports template data overrides for customization.
Search Zendesk knowledge base articles by title, content, or topic.
Search Zendesk tickets using query parameters like status, priority, or text search.
Search Zendesk users by email, name, or custom fields.
Update an existing Zendesk ticket with new status, priority, or other fields.
Parameter descriptions are minimal and lack critical detail. Most params lack format hints, range constraints, or dependencies. E.g. 'query' parameters in search_tickets, search_users, search_articles have single-line descriptions with no guidance on query syntax, length limits, or examples of valid queries.
Mutating tools (create_ticket, create_user, update_ticket, add_ticket_comment) lack idempotency guarantees, dry-run support, or confirmation patterns. No documentation of whether repeated calls with identical input produce duplicate side effects.
No tool annotations visible. Tools marked as READ_ONLY or WRITE risk lack explicit readOnlyHint or destructiveHint declarations. LLMs cannot reliably infer safety from tool risk labels alone.
No scope declarations or permission gates visible. Tools that modify Zendesk state (create_ticket, update_ticket, create_user) lack documented permissions or authorization checks.
Nullable parameters (requester_email, priority, type in create_ticket; status, priority in update_ticket; user_id, email in get_user; page in list_organizations) lack clarity on whether they are optional, what their defaults are, or what happens when omitted.
Search and list tools return results without pagination guarantees. No visible limit parameter, page size control, or total count field. Large result sets will blow context windows.
Parameter relationships undocumented. E.g., get_user accepts both user_id and email as nullable, no description of what happens if both are provided, or which takes precedence.
Demo tools (demonstrate_overrides) lack sufficient context for LLM selection. Description is vague about what 'template data overrides via double underscore notation' means and when an agent should use this tool.