Loading ToolBench
Motion understanding for videos and games. Analyzes video motion into annotated images (flowcards) and machine-readable stats for debugging motion issues like stalls, stutters, teleports, and flicker.
Four tools with complete schemas and detailed descriptions. Naming is action-oriented (record, analyze, compare, grab_frames). Descriptions are comprehensive (150-350 chars) and include WHEN/WHAT/NEXT guidance. All parameters have types and descriptions. However, no input validation constraints (enums, ranges) are enforced in schemas; error handling is basic (ToolError wrapping); no idempotency hints or permission gates; output schemas documented informally in text rather than structured JSON Schema. Composition is sound, tools chain naturally (record → analyze → compare). Risk annotations present but not formalized as tool annotations.
Compress a video's motion into ONE annotated image (card_path) + exact stats. The card answers: where did things move, when, in which direction, what flickered, what paths objects took, plus a timeline with STALL / APPEARANCE (teleport) / SPIKE markers and frame-pacing numbers. Camera motion is detected, removed from the panels, and reported separately (set camera_comp=False to see raw screen-space motion instead). Drill down by re-calling with t0/t1 (seconds) and/or roi=[x,y,w,h] (pixels) around a finding. Clips longer than ~18s are analyzed in a first-12s window automatically (the result carries a note telling you how to cover the rest). NEXT STEP: Read the returned card_path image.
Visual regression diff of two same-start recordings (golden vs current). Returns divergence onset time, the worst-diff region bbox, and a diff card (WHERE THEY DIFFER heatmap + WHEN THEY DIVERGE curve + A/B filmstrips). Both videos must share resolution and starting point - record them the same way. Use after a change to see if/when/where rendering behavior moved. NEXT STEP: Read the returned card_path image.
Return real video frames (JPEG) at the given timestamps in seconds. Max 6 timestamps per call. Use after reading the flowcard to see what things actually look like at event times.
Record the screen (or one window) to mp4 and, by default, immediately analyze its motion. Use this to capture a running game/app for debugging. target is 'desktop' (full screen - most reliable for games) or 'window:<exact title>' (gdigrab needs the exact, case-sensitive window-title text). seconds is clamped to [0.5, 30], fps to [10, 120] - capture at >= the app's frame rate to catch stutter. NEXT STEP: Read the returned card_path image.
No enum constraints on string parameters. 'target' in record() accepts 'desktop' or 'window:<title>' but no enum enforces this; LLM may hallucinate invalid values like 'monitor' or 'fullscreen'.
Numeric ranges documented in text but not in JSON Schema. 'seconds' clamped to [0.5, 30], 'fps' to [10, 120], 'max_width' to 960, no minimum/maximum fields in schema. LLM cannot parse constraints programmatically.
Output schemas documented only in docstrings, not as structured JSON Schema. Agents cannot parse expected fields (card_path, stats, panels_dir) from tool definition alone; must infer from description text.
Error handling returns ToolError with string messages but no recovery guidance. 'cannot open {video_path}' tells LLM the file is missing but not what to do next (e.g., 'Call record() to capture a video first').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). 'record' is destructive (writes files), 'analyze' and 'compare' are read-only, 'grab_frames' is idempotent, but these are not declared in tool metadata.