Loading ToolBench
open Platform for Agentic Simulation and Optimization. An MCP server that lets an AI model operate nine finite-element and multiphysics codes: simulation, cross-code coupling, verification and solver development.
openPASO shows reasonable naming and descriptions but suffers from critical gaps in schema visibility, parameter documentation, and output specification. Of 12 tools, schema information is inferred from source file references rather than directly visible in the tool registration code provided. Tool definitions appear distributed across multiple files (webui/static/assets/index-B8UnvA5p.js, src/core/pitfall_index.py, src/tools/consolidated.py) but no explicit schema registration is shown in the source excerpt. Descriptions are present and reasonably specific (60-150 chars, meeting the rubric minimum), but input schemas lack formal JSON Schema type/constraint declarations visible in the code. Parameters are documented but lack type constraints, enums, and validation guidance. No output schemas are documented for any tool. Error handling is not evident in the provided source. The server aims at a complex domain (FEM simulation coupling) but the tool definitions do not reflect the specificity required for LLM-safe interaction with destructive operations like 'run', 'couple', 'mesh', 'solver', and 'setup'.
Set up and manage cross-code coupling (e.g., FSI, TSI) between two backends
List available FEM backends and their capabilities, optionally filtered by query
Query and manage the session journal of all commands executed and their outcomes
Query knowledge base for a backend (pitfalls, postmortems, setup guides, physics descriptions, generator docs, materials, solver configuration)
Query and configure material properties and constitutive models
Load, validate, refine, and convert mesh files between formats
Extract, visualize, and analyze simulation results (fields, error estimates, convergence data)
No visible JSON Schema with type/constraint definitions for tool inputs. Parameters documented in descriptions but lack formal schema constraints (enum, minLength, pattern, minimum/maximum). LLMs cannot reliably select valid values without explicit enum/constraint declarations.
No output schemas documented for any tool. LLMs have no contract for what fields to expect, how to chain tools, or what data types are returned. Critical for tools like 'discover' and 'knowledge' that must return structured backend/solver information for downstream tools like 'setup' and 'run'.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | <=2025-11-25 | v2 |
Probe installed FEM backends and update the local registry of available solvers and capabilities
Execute a prepared input file on the specified backend and return results
Generate a complete runnable input file for a backend and physics type
Configure solver settings (linear solver type, tolerances, preconditioners)
Check simulation output for NaN/Inf and convergence issues
Destructive operations (run, setup, couple, mesh, solver) lack confirmation/dry-run patterns. An LLM can accidentally execute a 10-hour FEM simulation on a mistyped solver or mesh file without any reversibility mechanism.
'parameters' fields in setup/solver are typed as generic 'object' with no constraints. LLMs will hallucinate invalid physics parameters (e.g., 'density: infinity') that fail silently or produce NaN results. Physics-specific parameter validation (min/max, units, dependencies) must be documented or validated.
No error handling guidance visible. Tools like 'run' and 'couple' will fail (compilation errors, convergence failure, solver crash), but no error recovery patterns documented. LLM cannot know: is this retryable? Should I adjust parameters? Ask the user?
Parameter dependencies undocumented. E.g., 'mesh_file' in setup is required, but is it always? What if mesh is generated inline? 'output_format' in mesh action only applies to 'convert', using it with 'load' will be ignored or error. These must be explicitly declared.
Solver/physics identifier format unconstrained. 'solver' param accepts 'kratos', 'fenics', 'ngsolve' but also 'FEniCS', 'KRATOS', 'dolfinx'? Case sensitivity? Aliases? Without enum declaration, LLM guesses or uses examples literally.
'action' parameters in mesh and journal tools document valid values in description text only. Missing enum declaration in schema forces LLM to parse prose for valid options and risks hallucinated values like 'optimize' or 'compress'.
Tool composition chain unclear. 'discover' returns available backends, but does the response include the exact 'solver' parameter value for 'setup'/'run'? If not, LLM must chain discover → knowledge → (infer solver name) → setup, risking mismatches.
Tool registration code not visible in provided source. Tools inferred from file references and descriptions but no explicit `register_tool()` or schema definition shown. Scoring capped because actual implementation cannot be verified.