MCP server for efficient AI workflow orchestration with RelayPlane
RelayPlane MCP server has 10 tools with basic registration and Zod schema validation. However, critical gaps prevent higher scoring: (1) Tool descriptions are generic and lack LLM-optimized detail (e.g., 'Execute a single AI model call' does not explain WHEN to use vs relay_workflow_run). (2) Parameters lack type constraints and enums (e.g., 'language' in relay_exec_script accepts 'python', 'javascript', 'bash' but no enum declared; 'model' accepts freeform strings instead of enum of available models). (3) No output schemas are documented, LLMs cannot plan downstream calls. (4) relay_exec_script is an IRREVERSIBLE operation with no dry-run, confirmation, or rate-limiting mentioned. (5) Parameter descriptions are minimal (1 - 2 short phrases); LLM-optimized descriptions should be 50 - 200 chars explaining purpose and constraints. (6) Error handling is generic (switch/default case throws McpError); no recovery guidance or actionable error messages. (7) Composition issues: relay_run and relay_workflow_run overlap in purpose; unclear when to use each. (8) No pagination guidance for relay_runs_list (accepts limit/offset but no total_count or next_cursor documented). Average per-tool score: 42 across 10 tools.
Execute a script or code snippet with specified runtime
List available AI models with capabilities and pricing
List available execution policies and approval workflows
Execute a policy or approval workflow
Execute a single AI model call with specified prompt and model
Get detailed information about a specific workflow run
List previous workflow runs with status and metadata
List available pre-built workflow skills
IRREVERSIBLE operation relay_exec_script lacks confirmation, dry-run, or rate-limiting. No mention of recovery path if script fails mid-execution. Agents can execute arbitrary code (bash, python) without safeguards.
Tool descriptions are generic and too short (20 - 40 chars). LLM cannot distinguish when to call relay_run vs relay_workflow_run; no WHEN/WHY guidance. Descriptions should be 50 - 200 chars explaining purpose, preconditions, and differentiation.
No enums on constrained parameters. 'language' in relay_exec_script is freeform string; should declare enum [python, javascript, bash]. 'model' in relay_run accepts any string; should pull from relay_models_list output or declare known enum.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 35 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 31 | - | v1 |
Execute a multi-step AI workflow with sequential or parallel steps
Validate workflow syntax and structure without executing
Output schemas not documented. LLMs cannot plan downstream calls or extract required fields. relay_models_list output should list fields like 'model_id', 'provider', 'cost_per_token'; relay_run should document response structure for cost/duration/result.
relay_runs_list accepts limit/offset but no documentation of pagination behavior, total_count field, or next_cursor. LLMs cannot determine when to fetch more results or how many exist.
Parameter descriptions are minimal (1 - 2 phrases). 'The model ID to use (e.g., openai:gpt-4o)' lacks constraints on valid formats, character limits, or whether custom models are accepted. Descriptions should explain format, examples, and valid ranges.
Error handling is generic. CallToolRequestSchema catch block returns 'Tool execution failed: <message>' with no recovery guidance. LLM cannot determine if error is retryable, user-fixable, or fatal.
Composition overlap: relay_run and relay_workflow_run both execute AI models; unclear which to call first. relay_run is described as 'single AI model call' but relay_workflow_run is 'multi-step', a workflow with one step should use which? Tool descriptions do not clarify boundaries.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). relay_exec_script should be marked destructive; relay_models_list should be marked read-only. Tool definitions in src/server.ts hardcode switch/case matching; no structured metadata available to clients.