Loading ToolBench
Agent-first workflow orchestration engine with MCP support for executing workflows, managing templates, querying execution state, and generating workflow diagrams
Opcode is a workflow orchestration server with 6 tools covering execution, status tracking, signaling, template definition, querying, and visualization. Strengths: all tools have descriptions (100-200 chars, aligned with rubric baseline of 194 chars); input schemas present with types and enums for constrained parameters (signal_type, resource, format); explicit permission checks via agent_id; comprehensive error handling returning structured errors. Weaknesses: parameter descriptions are minimal (many under 50 chars when rubric baseline is 72); output schemas are documented only in descriptions, not in formal schema declarations; tool names lack consistent verb_noun patterns (e.g., 'opcode.run' is ambiguous, does it run a workflow or a script?); no enum constraints for resource_type in query tool despite using them elsewhere; error responses documented but recovery guidance sparse; composition could be improved, opcode.run + opcode.signal + opcode.resume are a three-call sequence that could be wrapped; no pagination on opcode.query despite it potentially returning large result sets.
Register a reusable workflow template. Version auto-increments (v1, v2, v3...)
Generate a visual diagram of a workflow. Returns ASCII art, Mermaid flowchart syntax, or base64-encoded PNG image
Query workflows, events, or templates. Returns {"<resource>": [...]} where key matches the queried resource type
Execute a workflow from a registered template. Returns workflow_id, status, and per-step results. Workflows with reasoning steps will return status 'suspended'
Send a signal to a suspended workflow. Decision signals automatically resume the workflow and return the final status
Get workflow execution status. Returns steps, events, and pending_decisions for suspended workflows
Tool names lack clarity. 'opcode.run' is vague, an LLM cannot infer whether it runs a workflow, a script, or a template without reading the full description. Rename to 'workflow.execute' or 'template.run' to match verb_noun convention.
Parameter descriptions are sparse. Most parameters have 1-2 sentence descriptions when rubric baseline is 72 chars. E.g., 'agent_id' in opcode.run says 'ID of the agent initiating the workflow' but does not explain format, where to find it, or whether it must pre-exist. Add 'Agent ID format: UUID or username; if not registered, system will auto-register.'
opcode.query lacks pagination controls. Accepting only filter without limit/offset/page parameters means agents cannot safely iterate large result sets. Add 'limit' (default 20, max 100) and 'offset' parameters, and return 'total_count' and 'has_more' in response.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 61 | 2025-06-18+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Output schemas are undocumented in formal schema declarations. Descriptions mention 'Returns workflow_id, status, and per-step results' but no JSON Schema is provided for tool outputs. LLMs cannot plan downstream tool calls without knowing exact response structure. Add outputSchema to tool registration.
No enum constraint for 'resource' in opcode.query. Despite supporting only ['workflows', 'events', 'templates'], the parameter is type 'string' without enum. Use enum: ["workflows", "events", "templates"] to prevent hallucinated resource types.
Composition inefficiency: opcode.run followed by opcode.signal followed by opcode.resume is a three-call pattern. For decision-suspended workflows, consider wrapping into a single 'workflow.decide' tool that runs, suspends, accepts decision, and resumes atomically.
Error responses are described as returning 'ToolResultError' but no structured error schema is defined (e.g., error_code, retryable flag, suggested_next_action). Pattern:recovery-guide recommends 'Try search_users() with a partial name', add actionable recovery hints.
agent_id is treated as optional or dynamically captured in opcode.signal ('if agentID != ""') but required in opcode.run. Inconsistency forces LLMs to reason about when to pass it. Make required across all tools or provide clear defaults.