Loading ToolBench
Server has 6 tools with complete input schemas and descriptions, but quality is uneven. Tool names are action-oriented (run_*, generate_*, validate_*, format_*, screenshot_*), which is good. Descriptions are present and moderately detailed (100-250 chars), meeting baseline minimums. However, parameter descriptions lack specificity on constraints, valid ranges, and error recovery guidance. Output schemas are documented in docstrings but not formally declared in the tool registration. Error handling returns structured dicts with 'success' and 'error' fields, but lacks actionable recovery guidance. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels (WRITE vs READ_ONLY). Missing enum constraints for theme and shell parameters, forcing LLMs to guess valid values.
Format a recording log as markdown text for attachment or sharing. Takes the dict returned by run_script or run_command and produces a clean markdown document with each command and its output in fenced code blocks — ready to paste into docs, issues, or Claude messages.
Generate a .reterm script from a list of commands. This tool helps AI create recording scripts programmatically.
Execute a single shell command and return structured output. This is a convenience tool for running one command without writing a full script. Useful for quick command execution with full output capture.
Execute a .reterm script and return the structured log. This is the primary tool for recording terminal sessions. The script uses YAML format to define commands, typing animations, and assertions.
Run a script and return a PNG screenshot of the terminal state. Executes the script and renders the terminal as a pixel-perfect image.
Missing enum constraints for 'theme' and 'shell' parameters across all tools. LLMs cannot infer valid values (dracula, nord, monokai, etc.) from free-form string descriptions. Should declare theme as enum with available options from list_themes().
No tool annotations (readOnlyHint/destructiveHint/idempotentHint) despite clear risk levels. run_script and run_command are marked WRITE (destructive), but tool definitions lack destructiveHint annotation. Agents cannot infer safety implications.
Error responses lack actionable recovery guidance. When script parsing fails or execution errors occur, responses return {'success': false, 'error': '...'} but do not suggest next steps (e.g., 'Check YAML syntax' or 'Verify shell availability'). Agents cannot self-correct.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Validate a .reterm script without executing it.
Parameter 'timeout_seconds' in run_command is documented as 'not yet implemented' but still exposed. This invites LLMs to pass a value that will be silently ignored, causing confusion. Either implement the feature or remove the parameter.
Output schemas documented in docstrings but not formally declared in tool registration. LLMs cannot parse docstring examples to infer response structure. Formal schema declarations (via fastmcp metadata or response type hints) would enable better planning.