Loading ToolBench
Deterministic physics tools for SiN photonic waveguide design. Provides eigenmode solving (modesolverpy), gradient-based inverse design (SAX + JAX), and GDSII mask generation (gdsfactory). Backed by a hybrid storage layer: DynamoDB simulation cache, S3 Express One Zone Parquet bulk data, and S3 Tables SQL queries.
Server has 11 tools with reasonable descriptions (avg 150-200 chars) and mostly complete input schemas. However, critical gaps undermine quality: (1) Tool naming lacks consistent verb-noun patterns, 'solve_waveguide_mode', 'optimize_waveguide', 'generate_mask' are clear, but 'query_dataset', 'query_low_loss_waveguides', 'read_dataset_columns' are redundant and confusing; (2) Output schemas are NOT documented, no tool specifies what fields are returned, forcing LLMs to guess; (3) Parameter descriptions lack constraint details (e.g., 'width_um' says 'e.g. 1.5' but no min/max stated in description, only in schema); (4) Error handling is absent, no recovery guidance or actionable error messages visible; (5) Two tools ('generate_mask' and 'generate_foundry_mask') appear to do the same thing, violating single-responsibility principle. Tools like 'plot_chart' are generic and lack domain context. Per-tool scores range 35 - 65; average 52.
Generate a GDSII mask file for the finalized waveguide design using gdsfactory. Produces a foundry-ready layout with tapers, routing, and I/O couplers.
Generate a foundry-ready GDSII mask file from waveguide parameters. Uses gdsfactory to create a photonic layout with proper routing, tapers, and I/O couplers. Returns the file path to the generated .gds file. GDSII file paths are cached in DynamoDB — if the exact same layout was previously generated, the cached file path is returned immediately.
Optimize waveguide geometry using gradient-based inverse design via SAX + JAX. Uses automatic differentiation to find the optimal width and height that achieve the target metric value.
Perform gradient-based inverse design to optimize waveguide geometry. Uses SAX (JAX-based photonic circuit solver) with automatic differentiation to find the optimal width and height that achieve the target metric value. Results are cached in DynamoDB — identical optimization requests return the previously computed optimum instantly.
Generate a chart from data.
Output schemas not documented. No tool specifies what fields are returned (e.g., solve_waveguide_mode returns n_eff, confinement_factor, MFD, group_index but this is only in description, not in a structured output schema). LLMs cannot plan downstream tool calls without knowing response structure.
Duplicate tools: 'generate_mask' and 'generate_foundry_mask' appear to perform identical operations (both generate GDSII files with same parameters). Violates single-responsibility principle and forces LLM to choose between functionally equivalent tools.
Query tools lack pagination. 'query_dataset' and 'query_low_loss_waveguides' accept a 'limit' parameter but do not return a total_count or next_cursor. With a 90K-row dataset, agents cannot iterate through results safely.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Predict propagation loss using XGBoost (fast first-pass estimate). For exact physics, use the MCP server's solve_waveguide_mode tool instead.
Query the 90K-row waveguide dataset with optional filtering and aggregation. Uses S3 Tables with predicate pushdown when available — filters are evaluated at the storage layer so only matching rows are read into memory.
Find waveguides below a propagation loss threshold.
Read specific columns from the dataset using Parquet columnar reads.
Solve waveguide eigenmodes using modesolverpy via the MCP physics server. Computes effective refractive index, confinement factor, mode field diameter, and group index using fully vectorial eigenmode expansion.
Compute waveguide eigenmode properties using fully vectorial EME. Returns effective refractive index (n_eff), optical confinement factor, mode field diameter (MFD), and group index for the specified waveguide cross-section and operating conditions. Results are cached in DynamoDB — repeated calls for the same geometry return in sub-10ms without re-running the eigenmode solver.
Parameter constraints in descriptions are incomplete. 'width_um' has gt:0 in schema but description says 'e.g. 1.5' without stating typical range. 'num_modes' says '1-10' in description but constraint is in schema, inconsistent. LLMs cannot reliably infer valid ranges from examples.
No error handling guidance. Tools provide no recovery hints (e.g., 'If optimization fails to converge, try reducing learning_rate or increasing max_iterations'). Agents cannot self-correct on failures.