Loading ToolBench
28 tools with mixed quality. Naming is verb-forward and clear (get_*, set_*, send_*, etc.), following pattern:tool conventions. Descriptions present but brief (avg ~40-60 chars, well below the 194-char baseline). All tools have input schemas with typed parameters and descriptions. However, output schemas are entirely undocumented, LLMs cannot predict return structure. No error handling guidance, no recovery paths. Parameter descriptions lack constraints (ranges, enums, formats). No tool annotations (readOnlyHint, destructiveHint). Risk classification visible in metadata but not in descriptions themselves.
Create a timestamped backup of the AdvancedOutput.xml file
Generate a diff between two AdvancedOutput.xml files
Export advanced output and slices XML files as a bundle
Get a summary of the advanced output configuration
Get a specific clip by layer and clip index
Get a specific column by index
Get the current composition state
Get a specific deck by index
No output schemas documented. LLMs cannot predict return structure, forcing them to guess what fields are available for downstream tool calls. This breaks tool chaining and increases hallucination risk.
Descriptions are too brief (avg 40-60 chars vs 194-char baseline). Missing context on WHEN to use each tool, prerequisites, and what happens on error. Descriptions like 'Trigger a parameter action' lack specificity, what action? What are side effects?
No error handling guidance. Tools like remove_parameter (DESTRUCTIVE) and restore_advanced_output_bundle (REVERSIBLE) lack recovery paths. LLMs cannot self-correct on failure or know whether to retry.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Get a specific layer by index
Get a parameter value by parameter ID
Get a summary of the slice inspector configuration
Load and parse the AdvancedOutput.xml preferences file
Load and parse the slices.xml preferences file
Post a parameter action
Preview what would be restored from an advanced output bundle
Remove a parameter
Rename a screen in the advanced output configuration
Rename a slice in the advanced output configuration
Reset a parameter to its default value
Restore advanced output configuration from a bundle
Save modified advanced output preferences to file
Send an OSC message to Resolume
Set the output device for a screen in advanced output
Set the vertices (corner points) of a slice in advanced output
Set the soft edge power value in advanced output
Set a parameter value by parameter ID
Trigger a parameter action
Get candidate paths for Resolume preferences on Windows
Parameter descriptions lack constraints. 'value' params accept JSON strings but no format/pattern guidance. 'parameter_id' accepts integers but no range bounds. LLMs will pass invalid values without validation hints.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Risk metadata exists in the server metadata but is not exposed in tool definitions. LLMs cannot determine which tools are safe to retry or have side effects.