Loading ToolBench
Agent-operated, MCP-first runtime for deterministic scripts in Docker. Zero LLM inside.
Dokan demonstrates solid tool design with 17 well-named, verb-prefixed tools covering script/flow/schedule/secret management. All tools have descriptions (avg ~120 chars) and input schemas with typed parameters. However, output schemas are not documented in the source, parameter descriptions lack detail on constraints/formats, and error handling guidance is absent. No tool annotations (readOnlyHint/destructiveHint) despite clear WRITE/READ_ONLY risk classifications. Naming is consistent and clear (upload_script, run_script, get_run, etc.), following verb_noun convention. Parameters are typed but many lack format/range constraints in descriptions.
Cancel a pending or running flow run.
Cancel a pending or running run.
Compose a declarative DAG of steps (each step = one container run). Supports map fan-out, when conditionals, and saga compensation.
Create a cron schedule that enqueues a run for a script on each tick (6-field cron with leading seconds).
Delete a secret by name.
Poll a flow run's status, step outputs, and structured results. Returns the flow run row and all step details.
Poll a run's status, output, logs, and structured result. Returns the run row and optionally log lines.
Output schemas not documented. Tools return data but LLMs cannot plan downstream calls without knowing response structure (e.g., what fields does get_run return? Does it include step_id for chaining to other tools?).
Parameter descriptions lack format/range constraints. E.g., 'mem_limit_mb' (integer) has no min/max; 'cpu_limit' (number) has no range; 'cron' (string) says '6-field' but no validation pattern. LLMs cannot infer valid ranges and may pass absurd values.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear WRITE/READ_ONLY risk labels in metadata. Agents cannot distinguish safe reads from destructive writes without explicit hints.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | <=2025-11-25 | v2 |
Retrieve a secret by name (value is masked in responses).
List all cron schedules with their enabled status.
List all uploaded scripts with optional search by name (substring or fuzzy via pg_trgm).
List all secret names (values are masked).
Stop and forget a schedule (remove the live job and disable it in the DB).
Enqueue a flow run. Returns immediately with a flow_run_id; the executor drives the DAG asynchronously.
Enqueue a script run. Returns immediately with a run_id; the executor claims and runs it asynchronously.
Semantic search over scripts using local BGE embeddings (if --embed is enabled) or substring fallback.
Store a secret (name-value pair) that is injected as an env var into every job. Values are masked in the UI/API.
Upload a script (idempotent). Stores source code for a runtime (python, node, bash, etc.) with optional network isolation and resource overrides.
Error handling guidance absent. No recovery hints (e.g., 'If script not found, call list_scripts() first'). Agents cannot self-correct on failures.
compose_flow 'spec' parameter is type 'object' with no schema detail. LLMs cannot validate DAG structure (steps, depends_on, map, when, retries, cache, compensate) without a formal schema.