Loading ToolBench
Numerically-grounded physics experiments as agent tools. Exposes the nls_torus experiment toolkit over the Model Context Protocol so an agent can run deterministic numerical experiments and reason over returned metrics and verification signals.
Server has 7 tools with clear names (list_, run_, sweep, compare, compose, search_, probe) and detailed descriptions (avg 180 chars). All tools have input schemas with types. However, several critical gaps: (1) compose and probe accept arbitrary Python code as strings, no validation, format constraints, or safety guardrails documented; (2) parameter descriptions lack format/range constraints (e.g., 'values' in sweep is just 'List of values', no guidance on numeric vs categorical); (3) output schemas are documented in descriptions but not formally specified in JSON Schema; (4) error handling is minimal, no recovery guidance or categorization; (5) compose/probe are WRITE operations but lack confirmation/dry-run patterns; (6) probe requires external API key (GOOGLE_API_KEY) but this is not mentioned in the tool description, only in a comment.
Run several labelled configs of the same experiment side by side. `configs` maps a label to a params dict. Returns each config's metric + verification + summary, and (for a numeric metric) which label extremises it.
Design and run a NEW experiment the registered ones don't express, with HARNESS-OWNED verification. `code` must define run(Nx) -> {"observable": <the answer>, "mass_series": [diagnostics.mass(U, surf["Mdiag"]) per step], "energy_series": [...], "metrics": {...}}. The server runs it at two resolutions in a subprocess sandbox and COMPUTES the trust flags itself (mass/energy conservation from the series, grid_converged from whether `observable` agrees across resolutions) — you cannot set them. Returns {metrics, verification, trustworthy}. The library is preloaded: build_surface, ring_grid, wg_chain, CNStepper, RingStepper, fields, diagnostics, bogoliubov, geometry.
List the available numerical experiments, their parameters and defaults.
Delegate to the generative ManifoldExperimenter (NOOA/Gemini sub-agent): it designs, runs, and HARNESS-verifies a NEW experiment to answer `question` on `geometry`, then returns {question, summary, metrics, verification, trustworthy}. Slow (LLM + compute) and needs a py3.12+nooa interpreter + GOOGLE_API_KEY (runs in a subprocess). For most cases use compose() instead.
compose and probe accept arbitrary Python code as 'code' parameter (string type, no validation). No format constraints, safety guardrails, or sandboxing documented in the tool description. LLMs could pass malicious or syntactically invalid code.
probe requires GOOGLE_API_KEY environment variable but this dependency is not documented in the tool description, only in a code comment. LLMs cannot know the tool will fail without the key.
compose and probe are WRITE operations (Risk: WRITE) but lack confirmation/dry-run patterns. Agents cannot preview results before committing expensive compute.
Parameter descriptions lack format/range constraints. 'values' in sweep is described as 'List of values to sweep over' with no guidance on numeric vs categorical, min/max length, or expected types. 'metric' defaults to empty string but no guidance on valid metric names.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Run one experiment. Returns {metrics, verification, provenance, summary}. The verification block (conservation drift, Wronskian residual, plateau match, ...) tells you whether to trust the numbers.
Search arXiv (title/abstract/authors) to cross-check a numerical finding against the literature or find the relevant paper — the Townes critical mass, a Kibble-Zurek exponent, the analog-Hawking temperature. Returns {id, title, authors, summary, url} per hit. No API key. The RUN is the evidence; use this only to sanity-check.
Scan one parameter over `values`, holding `base_params` fixed. Returns the chosen `metric` (and full metrics + verification) per value, plus a log-log scaling fit (exponent) when both axes are positive numeric — e.g. Kibble-Zurek k* ~ tau_Q^b.
Output schemas are documented in tool descriptions (e.g., '{metrics, verification, provenance, summary}') but not formally specified as JSON Schema. LLMs cannot reliably parse or validate responses.