Loading ToolBench
Infrastructure for safely using OpenMC with LLMs: plan, build, validate, run, analyze
PromptMC provides 10 tools with complete input schemas and descriptions. Naming follows verb_noun convention (check_, validate_, render_, list_, run_, analyze_, geometry_debug, cross_section_check, plot_). Descriptions are present and substantive (avg ~120 chars), exceeding the 10-char minimum. However, several parameter descriptions lack specificity about constraints, ranges, and error recovery. Output schemas are defined via Pydantic models but not all are visible in the provided code excerpt. Error handling exists but lacks actionable recovery guidance. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are present. The server demonstrates solid foundational quality but falls short of A-grade polish in parameter documentation and error messaging.
Parse simulation outputs into a structured summary.
Report the local OpenMC installation status.
Check OpenMC cross-section data availability and configuration.
Debug OpenMC geometry by checking for overlaps and other issues.
List the available configuration templates.
Generate a plot of the OpenMC geometry.
Render a complete OpenMC input deck from a named template. Writes settings.xml, geometry.xml and materials.xml into the requested output directory.
Parameter descriptions lack constraint documentation. E.g., 'threads' in run_simulation has no min/max bounds; 'pixels' in plot_geometry lacks resolution limits; 'particles', 'batches', 'inactive' in render_template lack valid ranges. LLMs will pass arbitrary values without guidance.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present. Agents cannot distinguish safe read-only tools from destructive ones (render_template, run_simulation, plot_geometry write files). This violates current MCP spec (2026-07-28) which expects tool annotations for risk classification.
Error handling lacks actionable recovery guidance. Tools return errors but do not suggest next steps. E.g., if validate_input fails, should the agent call schema_check? If run_simulation times out, should it retry with fewer threads? Errors must guide agent recovery.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 71 | 2026-07-28+ | v2 |
Run an OpenMC simulation and capture the result.
Run Pydantic schema validation over OpenMC input files.
Validate an OpenMC XML input file or directory.
Parameter descriptions for 'mode' in run_simulation and 'basis' in plot_geometry are minimal. 'mode' enum values (auto, api, subprocess) lack explanation of when to use each. 'basis' enum (xy, xz, yz) lacks context about coordinate system orientation.
Output schemas (AnalysisResult, SimulationRunResult, etc.) are defined in schemas.py but not visible in provided code. Cannot verify whether outputs include chaining IDs (e.g., does analyze_results return paths needed by downstream tools?) or whether results are paginated/capped.