Loading ToolBench
MCP server driving the Learning-to-Swim runbook on RunPod — manages pod lifecycle, training jobs, and GPU resource allocation for two vehicles (hippocampus and bluerov2)
14 tools with mostly complete schemas and descriptions, but significant gaps in error handling guidance, parameter constraints, and output documentation. Tool names are action-oriented (ensure_, run_, apply_, sync_) and descriptions are detailed (avg 180 chars), but lack LLM-optimized recovery hints. Parameters have types and descriptions, but many lack enums/constraints for constrained inputs (e.g., vehicle, dr_level). Output schemas are not formally documented. Dry-run patterns and confirmation gates are present (terminate_pod requires verbatim confirm string), showing security awareness. However, no tools document return field structures or pagination. Error handling is minimal, most tools raise ToolError but don't guide recovery.
Automate BLUEROV2/patches/APPLY.md §§1-5 on the BLUEROV2 pod (bluerov2 by nature — no vehicle param): restore pristine 7c5ebe7, install the 8-thruster drop-in, apply the content-anchored edits, write markers/bluerov2_patch_applied. Idempotent (pristine-then-apply). Refuses while a job is live unless force=True (the checkout is shared).
Run BLUEROV2/patches/axis_sanity_sweep.py headless as a detached job on the BLUEROV2 pod (bluerov2 by nature — no vehicle param); writes markers/axis_sanity_PASS ONLY on exit 0. MANDATORY after patching and BEFORE any bluerov2 training (RUNBOOK Days 4-5).
Ensure this vehicle's pod is RUNNING and SSH-ready: network volume first, then resume-or-create (4090/SECURE/non-interruptible enforced in code). vehicle: hippocampus (pod lts-replication) | bluerov2 (pod lts-replication-bluerov2) — each has its own volume, known_hosts file and log dir. Refuses if any UNDECLARED pod exists (the other vehicle's pod is fine). Polls ~5 min, verifies the driver floor, truncates known_hosts and re-arms the idle watchdog on every transition-to-running. dry_run returns the exact would-be payloads.
Bounded synchronous SSH escape hatch (ceiling 600s — use run_job for anything longer). Unfiltered by design: guardrails protect money, not the pod filesystem.
No output schemas documented. Tools return dicts (job_id, max_runtime_sec, etc.) but LLMs cannot see field types or structure. Downstream tools cannot reliably extract required IDs for chaining.
Constrained inputs lack enums. 'vehicle' accepts 'hippocampus'|'bluerov2' but is typed as free-form string. 'dr_level' accepts 'DR_0/1/2' or 'none/small/large' but no enum constraint. LLMs may hallucinate invalid values.
Error handling lacks recovery guidance. ToolError exceptions are raised but do not tell LLMs what to do next (e.g., 'run ensure_pod first', 'retry in 30s', 'check pod logs'). Agents cannot self-correct.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
RTX 4090 price + stock via RunPod's public GraphQL (read-only, $0). Optionally filter to one datacenter (e.g. 'EU-RO-1', where the network volume lives). vehicle: hippocampus|bluerov2 (which pod's config to use).
State of a detached job (running/succeeded/failed/orphaned/not_found) + log tail + latest 'Mean reward' line. exit_code 124 = wall-clock ceiling hit.
One RUNBOOK training run (vehicle: curee|bluerov2; dr_level: DR_0/1/2 or none/small/large). The vehicle also picks the pod: curee trains on the hippocampus pod (lts-replication), bluerov2 on lts-replication-bluerov2. Rewrites the two DR cfg lines content-anchored (aborts on unrecognized current values), then launches the verbatim RUNBOOK train command. Gates: bluerov2 needs patch+sanity markers; curee refuses on a patched checkout. Poll with job_status.
Pod state (no_pod / stopped / running / running_ssh_pending) + driver, disk, active jobs, $/hr for one vehicle's pod. Read-only.
Generic detached job (eval scripts, pretrained cross-check, plots). Refuses while another job is live on that pod unless force=True. auto_stop stops the pod when the job ends (probed at launch — fails loudly if it can't work).
Push runbook/pod_setup.sh verbatim and run it as a DETACHED job (non-interactive env: DEBIAN_FRONTEND, IsaacSim EULA consent). First run 30-60 min (5400s ceiling); re-run all-skips ~1-2 min (Day-1 drill). Poll with job_status.
Billing so far (pod compute + network volume storage) vs budget_usd. ACCOUNT-WIDE — RunPod billing has no per-pod filter, so this covers BOTH vehicles (no vehicle param). Informational only — the hard stop is RunPod's prepaid credit balance.
Default teardown — stop the pod whenever not training. vehicle is REQUIRED (no default): 'hippocampus' (lts-replication) or 'bluerov2' (lts-replication-bluerov2). Refuses if a job is live unless force=True. Idempotent. Stopped pods bill only volume storage.
rsync -az --partial pod:/workspace/IsaacLab/logs/<subdir>/ down to that vehicle's local_log_dir/<subdir> (hippocampus -> logs/pod, bluerov2 -> logs/pod/bluerov2; both gitignored). Never deletes on either side.
DELETE one vehicle's pod (its network volume survives). vehicle is REQUIRED (no default): 'hippocampus' or 'bluerov2'. Requires the verbatim confirm string 'terminate <that vehicle's pod_name>' — i.e. 'terminate lts-replication' for hippocampus, or 'terminate lts-replication-bluerov2' for bluerov2. Human-approval-only per the root CLAUDE.md.
Parameter relationships undocumented. launch_training's 'vehicle' determines which pod is used (curee→hippocampus, bluerov2→bluerov2), but this mapping is not stated in parameter descriptions. LLMs cannot reason about dependencies.
axis_sanity_sweep has minimal description (65 chars). Does not explain when to call it, what it validates, or what 'PASS' marker means. LLMs cannot determine if this is a prerequisite or optional step.