Open-source client SDK, MCP server, and agent Skills for the SwarmLabs physics-informed scientific experiment automation engine — including vv-gate, an independent fail-closed verification gate.
Server exposes 9 tools with basic schemas and descriptions, but significant gaps undermine production readiness. Tool names lack consistent verb-noun structure (e.g., 'vv_gate_scenarios' vs 'vv_gate_check'). Descriptions are present but generic (avg ~80 chars, below 194-char baseline). Parameter descriptions are sparse or missing entirely, 'params' in swarmlabs_run/sweep lacks type detail; 'predictions' in vv_gate_verify has minimal guidance. No output schemas documented. Error handling returns raw error objects without recovery guidance. No tool annotations (readOnlyHint, destructiveHint). Composition is reasonable (single responsibility per tool), but parameter naming inconsistencies (engine vs scenario_key) and missing constraints (no enum for engine IDs, no bounds on n) force LLM guessing.
List available engines and physics-model coverage.
Run a real physics-informed prediction for a SwarmLabs engine.
Parameter sweep for trend analysis (free tier).
Return the wet-lab anchor chain (second evidence chain), including where it contradicts the V&V chain.
Check the trustworthiness of a claim against the published gate ledger. Returns PROCEED, PROCEED_WITH_HUMAN_CHECK, or BLOCK_AUTONOMOUS_ACTION.
Return the full gate ledger with engine/oracle digests and provenance annotation.
List published verification-gate scenarios and their verdict/gate status.
Output schemas completely undocumented. Tools return JSON but LLMs cannot infer field structure, types, or chaining IDs. Forces agents to guess what fields exist and how to extract data for downstream calls.
Parameter 'params' (swarmlabs_run, swarmlabs_sweep) is type 'object' with no property schema. LLMs cannot determine what physics parameters are valid, required, or constrained. Needs nested schema with typed fields and descriptions.
Tool names inconsistent: 'swarmlabs_*' vs 'vv_gate_*' prefixes; 'scenarios' (noun) vs 'check' (verb). LLMs struggle to parse intent from mixed naming conventions. Standardize to verb_noun pattern (e.g., list_scenarios, check_gate_claim).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | 2026-07-28+ | v2 |
Hand out the held-out input skeleton for a scenario to enable prediction submission.
Verify your own predictions against the published held-out set. Returns R²/coverage/kappa plus fail-closed gate verdict (PROCEED | PROCEED_WITH_HUMAN_CHECK | BLOCK_AUTONOMOUS_ACTION).
Error handling returns raw SwarmLabsError objects with 'error' and 'status' fields. No recovery guidance (e.g., 'Try list_engines() first' or 'Invalid engine ID, available: vqe_h2, vqe_h3'). Agents cannot self-correct.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All tools are read-only but this is not declared. Agents cannot distinguish safe tools from destructive ones or optimize retry logic.