Loading ToolBench
Multi-module MCP server for ServiceNow governance across change management, CMDB, incident/problem management, and delivery/RAID tracking
Scoring was not performed
Duplicate tool name 'create_raid_item' with different signatures (3-param vs 6-param). Lines 13 and 1 in tools list both define create_raid_item but with incompatible parameter sets. This breaks tool selection, LLMs cannot determine which signature to use.
No output schemas documented. All 25 tools return Dict[str, Any] or similar generic dicts. LLMs cannot predict what fields will be present in responses, preventing proper chaining and data extraction. Every tool response needs a documented schema.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 22 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 53 | - | v1 |
All 25 tool implementations are TODO stubs with print() statements instead of real ServiceNow API calls. The actual behavior is undefined. Code like 'print(f"[SNOW] Change {change_id} approved")' does not interact with ServiceNow, comments say 'TODO: Replace with real ServiceNow call'. This makes the server non-functional.
Parameter descriptions are generic and lack LLM-actionable context. Examples: 'rationale' (9 chars), 'message' (7 chars), 'note' (4 chars). These do not explain WHAT the LLM should write or WHEN to use them. Baselines show effective descriptions are 50-200 chars with context about constraints, format, and decision criteria.
No error handling guidance. Tools return success dicts without addressing failure modes. If approve_change fails (e.g., change not found, no permission), what should the LLM do next? No recovery paths, retryable vs fatal classifications, or actionable error messages are provided.
Missing parameter constraints. Tools like update_ci accept 'fields': {"type": "object"} with no documentation of valid field names, types, or ranges. update_incident and update_raid_item have the same issue. LLMs will guess at field names and pass invalid payloads.
Tool descriptions lack decision context. 'Approve a change with documented rationale. Only call this if all governance checks pass.' does not explain WHAT governance checks to perform or HOW to verify them. The condition 'only call this if...' is aspirational but unsupported by the tool definition. An LLM cannot determine when this condition is met.
Tool composition is incomplete. Many tools (request_more_info, request_ci_owner_update) have no way to determine success or failure. Requesting info via work note provides no confirmation that the owner actually responded. The tool returns success regardless of outcome.
Idempotency is not addressed. Tools like add_ci_work_note, add_incident_work_note, add_problem_work_note offer no idempotency guarantees. If an LLM retries after a transient failure, duplicate work notes will be added.
No pagination or result limiting on list-style operations. While tools like 'get_change_cis' are implemented as resources rather than tools, if search/list tools are added in future modules, no limits or pagination mechanisms are visible. Baselines show this as critical for context window management.