Loading ToolBench
MCP server for interacting with Integrator automations
Single tool with critical definition gaps. Tool name 'run_scenario_*' uses generic verb 'run' (pattern violation). Description is minimal (scenario name + optional description, ~50 chars avg), below 10-char minimum for clarity. Input schema is dynamically generated at runtime from scenario.interface.input, making it impossible to validate statically. No parameter descriptions visible. Output schema undocumented. Error handling returns raw error strings without recovery guidance. Tool definition is inferred from code pattern matching (/^run_scenario_\d+$/) rather than explicit registration, triggering the 50-point cap for inferred tools.
Dynamically generated tools to run on-demand scenarios from Integrator. Tool name format is run_scenario_{scenarioId}. Description is the scenario name with optional description appended.
Tool name 'run_scenario_*' uses generic verb 'run' instead of action-specific verb. LLMs cannot infer intent from 'run' alone, does it execute, test, simulate, or trigger? Rename to 'execute_scenario_*' or 'trigger_scenario_*' for clarity.
Input schema is dynamically generated at runtime from scenario.interface.input. No static schema visible in code. Cannot validate parameter types, required fields, or constraints. Violates pattern:constrained-input, LLMs will hallucinate invalid parameter values.
Tool description is only scenario name + optional description (~50 chars). Does not explain WHAT the tool does, WHEN to use it, or what it returns. Violates pattern:tool-description. LLMs cannot determine when to select this tool vs alternatives.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 26 | 2026-07-28+ | v2 |
Output schema is undocumented. Tool returns JSON.stringify(output) or 'Scenario executed successfully.', no schema definition for LLM to plan downstream calls. Violates pattern:response-shaper.
Error handling returns raw error strings (String(err)) without recovery guidance. LLM receives 'Error: Invalid input' with no hint to retry, ask user, or call a discovery tool. Violates pattern:recovery-guide.