Loading ToolBench
ZOS-API automation, typed tools, and MCP server for optical design optimization via Zemax OpticStudio
OptiVibe MCP exhibits significant definition quality gaps. Tool descriptions are present but minimal (10-50 chars, well below the 50-200 char LLM-optimized baseline). Input schemas are sparse: most tools accept only a 'config' parameter with minimal type info. No output schemas are documented. Parameter descriptions are generic ('Configuration number') without explaining constraints, valid ranges, or error conditions. The 30-tool set lacks clear composition patterns, many tools operate on the same 'config' resource without distinguishing when to use analyze_strehl vs analyze_wavefront vs analyze_distortion. Error handling is absent from visible definitions. No tool annotations (readOnlyHint, destructiveHint) despite clear risk stratification (READ_ONLY, WRITE, DESTRUCTIVE). Naming is verb-forward (get_, analyze_, set_, add_, remove_) but lacks specificity, 'analyze_*' tools do not explain what metric they return or how to interpret results.
Add a mathematical constraint
Add an operand to the merit function
Analyze axial color aberration
Analyze optical distortion
Analyze GRIN (gradient-index) profile
Analyze lateral color aberration
Analyze relative illumination
Minimal parameter descriptions. Most tools accept 'config' with only 'Configuration number or all for all configurations', no explanation of valid range, error behavior, or when to use 'all' vs a specific number.
No output schemas documented. Tools like analyze_strehl, get_mtf, get_spot, render_layout return unspecified data structures. LLMs cannot plan downstream calls or extract required fields without knowing response shape.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | 2026-07-28+ | v2 |
Analyze Strehl ratio
Analyze wavefront aberrations
Check optical clearance
Describe optical surfaces
Find pairs of glass materials matching specified criteria
Find glass materials matching specified criteria
Get first-order optical properties
Get modulation transfer function
Get operand value from the optical system
Get spot diagram data
Retrieve system information from the ZOS-API session
Look up glass material specifications from the glass catalog
Look up operand specifications from the reference catalog
Run optical optimization
Remove a configuration
Render optical layout diagram
Search the reference documentation and operand database
Set configuration parameter value
Set configuration variable
Set the current active configuration
Set vignetting parameters
Trace rays through the optical system
Verify collimation of the optical system
No tool annotations despite clear risk stratification. Tools marked DESTRUCTIVE (remove_configuration) or WRITE (set_vignetting, optimize) lack readOnlyHint/destructiveHint/idempotentHint annotations. Agents cannot distinguish safe from dangerous operations.
Ambiguous analyze_* tool set. Six tools (analyze_strehl, analyze_wavefront, analyze_distortion, analyze_relative_illumination, analyze_lateral_color, analyze_axial_color, analyze_grin_profile) all accept the same 'config' parameter with identical descriptions. No guidance on when to use each or how results differ.
Missing error handling guidance. No descriptions explain what happens on invalid config numbers, closed sessions, or API failures. Agents have no recovery path.