Loading ToolBench
MCP Server — expose workflow operations through MCP protocol with 6 tools for workflow execution, validation, planning, composition, and discovery
The agency-orchestrator MCP server exposes 6 tools with mixed quality. Tool names follow verb_noun convention (run_workflow, validate_workflow, list_workflows, plan_workflow, compose_workflow, list_agents), which is positive. However, descriptions are minimal (most under 100 chars), parameters lack depth in documentation, and output schemas are entirely undocumented. The schema definitions visible in the code show basic type information but no constraints, enums, or format specifications. Error handling and recovery guidance are absent from the tool definitions. While the tool set is coherent in purpose (workflow orchestration), the definitions lack the LLM-optimization rigor expected of production-grade tools. No tool annotations (readOnlyHint, destructiveHint) are present despite clear risk classifications (run_workflow and compose_workflow are WRITE operations). Input validation rules, parameter ranges, and output structure documentation are missing across all tools.
Generate a workflow YAML from a natural language description using AI
List available AI agents/roles from the agents directory
List available workflow templates from the workflows/ directory
Show the DAG execution plan for a workflow
Execute a YAML workflow with the DAG engine
Validate a workflow YAML without executing
Output schemas are completely undocumented. No tool returns a documented structure, making it impossible for LLMs to plan downstream calls or extract required fields for chaining.
Tool descriptions are below the 50-character LLM-optimization baseline for most tools. 'Execute a YAML workflow with the DAG engine' (45 chars) lacks context on WHEN to use this tool versus plan_workflow, and omits what the output contains.
No parameter constraints or validation hints. The 'provider' and 'model' parameters in run_workflow and compose_workflow accept free-form strings with no enum, pattern, or range documentation. LLMs will hallucinate invalid provider names.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
No tool annotations despite explicit risk classifications. run_workflow (WRITE) and compose_workflow (WRITE) lack destructiveHint annotations. validate_workflow, list_workflows, plan_workflow, list_agents (READ_ONLY) lack readOnlyHint annotations. This prevents agents from understanding operation safety.
Parameter descriptions are either missing or trivial. For example, 'inputs' in run_workflow is described as 'Key-value input variables' but does not specify: required vs optional keys, data types (string, number, boolean, array?), example structure, or constraints.
No error handling guidance. If run_workflow fails due to invalid YAML, missing file, or LLM provider error, there are no documented recovery hints (e.g., 'check YAML syntax with validate_workflow first', 'verify provider API key').
Tool composition clarity is unclear. The relationship between plan_workflow (shows DAG) and run_workflow (executes) is not documented. Should LLMs always call plan_workflow first? When should they skip it? This forces the agent to guess.
No pagination or result-limiting documented. list_workflows and list_agents do not specify: max results returned, pagination mechanism, or whether results are truncated if the directory contains thousands of items.