Simple, flexible workflow orchestration for multi-agent AI applications
AgentLoom exposes 16 tools with basic schemas and descriptions, but critical gaps undermine production readiness. Most tools lack parameter descriptions (e.g., workspace, relative_path, timeout_seconds have no guidance on format or constraints). Descriptions are present but generic, many under 100 chars and lacking WHEN/WHY context. No output schemas documented. Error handling is absent; tools do not guide recovery. Tool names are reasonable (verb_noun pattern) but lack clarity on mutual exclusivity and dependencies. No evidence of idempotency, rate limiting, or permission gates. STDIO transport caps protocol score at 50.
Run `browser-harness --doctor` and return structured diagnostics.
Generate directory-mirrored Markdown code map files from scan results.
Report whether persisted parallel evidence belongs to the current Goal.
List the controlled repository's source, configuration, tests and contract.
Load a bounded, line-numbered evidence bundle for one audit topic.
Return a large JSON list with one hidden validation item.
Return a large log-like payload with one hidden validation record.
Parameter descriptions missing or trivial. workspace, relative_path, timeout_seconds, script, browser_mode, case_id, topic, tasks_json, concurrency, output_dir lack guidance on format, constraints, or valid ranges. LLMs cannot infer parameter meaning from names alone.
No output schemas documented. Tools return unstructured results (e.g., 'structured diagnostics', 'evidence bundle', 'report'). LLMs cannot plan downstream calls or extract required fields without knowing response structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
Return a large grep-like payload with one hidden validation match.
Return a large plain-text payload with one hidden validation record.
Atomically persist one batch report or final summary.
Read one actual source, configuration, test, contract or report file.
Run Python code through browser-harness via stdin.
Run a validated real-LLM audit batch through AgentLoom's parallel API.
Run parallel contract Workers and persist evidence for the current Goal.
Actually execute every existing and generated pytest test, retaining evidence.
Write model-authored source, generated regression tests or a structured report.
No error handling guidance. Tools do not indicate what errors are retryable, user-fixable, or fatal. No recovery suggestions (e.g., 'If file not found, call inspect_workspace first'). Agents cannot self-correct on failure.
Destructive operations (write_workspace_file, run_workspace_tests, run_goal_audit_batch, persist_goal_validation_report) lack confirmation or dry-run support. Agents can accidentally overwrite files or corrupt state without safeguards.
Tool descriptions lack WHEN/WHY context. 'Return a large JSON list with one hidden validation item' does not explain when to call make_context_engine_json_payload vs the log/search/text variants. Ambiguous descriptions force LLM guessing.