Loading ToolBench
Universal 'let the agent see your screen' tool — works as an MCP server (Claude Code, Cursor, Windsurf, Cline, Codex CLI) or a plain CLI (Aider, shell scripts, anything).
ScreenSight demonstrates solid tool design with clear, action-oriented naming (screen_*, all verbs) and comprehensive descriptions (avg 280 chars, well above 194 baseline). All 9 tools have documented input schemas with type definitions and parameter descriptions. However, output schemas are not formally documented in the code, descriptions mention return values but no structured schema definitions are visible. Error handling is mentioned in descriptions (e.g., 'refused and an error is returned') but recovery guidance is minimal. Tool composition is excellent: single responsibility per tool, clear chaining (enable→capture), and idempotent operations. Security considerations are well-documented (blocklist, redaction zones, audio gating). Missing: explicit enum constraints for some parameters (e.g., display index), formal output schema documentation, and actionable error messages.
Take a screenshot of the current screen. Captures the primary display (or a specific display by index) as a JPEG and returns the file path. The image is automatically downscaled to keep vision-model token usage reasonable, and any user-configured redaction zones are blacked out. The master switch must be ON (screen_enable) before this works. If the active window's title matches the sensitive-app blocklist (password managers, private-browsing windows, etc.), the capture is refused and an error is returned — the frame is never written to disk. Args: display: Display index (0 = primary, 1 = secondary, etc.). Omit or pass None for the primary display. question: Optional question to echo back for your context. Returns: File path, SHA256 hash, and active window title (if available).
Record the system's audio output (what's playing through the speakers). Captures loopback audio — the sound the user is hearing (music, a video, a call) — for `duration` seconds and returns it as an audio content block you can listen to, plus text context. This is separate from screen_capture: audio is time-based, so this call blocks for roughly `duration` seconds while recording. Two gates must be satisfied: - The master switch must be ON (screen_enable). - Audio must be enabled by the user via the SCREENSIGHT_ENABLE_AUDIO=1 environment variable — it is OFF by default for privacy. Platform notes: Windows (WASAPI) and Linux (PulseAudio monitor) work out of the box; macOS and WSL require a virtual loopback device (BlackHole / SoundFlower) set as the default output. Args: duration: Seconds of audio to record (1–30, default 5). question: Optional question to echo back for your context. Returns: Audio content block + text context (+ echoed question if given).
Turn OFF the ScreenSight master switch. Disables all screen capture. Any running watch daemon will also stop. The frame file (if any) is NOT deleted by this call — use this when you want to temporarily pause captures. Returns the current state as JSON.
Output schemas not formally documented. Descriptions mention return values (e.g., 'File path, SHA256 hash, active window title') but no structured JSON Schema definitions are visible in the code. LLMs cannot reliably parse unstructured output or plan downstream tool calls without explicit schema.
Numeric parameters lack explicit constraints. 'display' (integer) and 'duration' (1 - 30) have ranges mentioned in descriptions but no min/max schema constraints. 'interval' and 'max_frames' have defaults but no bounds. LLMs may pass invalid values (negative, zero, or absurdly large numbers).
Error handling lacks recovery guidance. Descriptions state errors occur (e.g., 'capture is refused and an error is returned') but do not tell the LLM what to do next. No distinction between retryable, user-fixable, and fatal errors. Example: 'If sensitive app detected, try screen_disable then re-enable after closing the app.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Turn ON the ScreenSight master switch. The master switch must be ON before any screen capture can happen. This is a safety gate — nothing is captured while it's off. Call this before screen_capture if you're unsure whether it's on. Returns the current state as JSON.
List all available displays on this system. Returns a list of displays with their indices, names, and (on Windows) resolutions. Use the index with screen_capture's `display` parameter to capture a specific display. Args: question: Optional question to echo back for your context. Returns: List of display dicts with keys: index, name, (resolution on Windows).
Check whether ScreenSight's master switch is on or off. Returns enabled/disabled status and when it was last changed. Call this to verify the switch state before attempting a capture.
Retrieve the most recent frame from a running watch session. If the watch daemon is running, returns the latest captured frame (if it has changed since the last call). If no daemon is running or no frames have been captured yet, returns an error. Args: question: Optional question to echo back for your context. Returns: File path, SHA256 hash, and active window title (if available).
Start a bounded watch session: poll the screen at intervals and collect changed frames until a limit is reached, then auto-stop. Useful for capturing a sequence of screen changes (e.g. a user interaction, a build process, a video playing) without manually calling screen_capture multiple times. The watch daemon runs in the background and writes frames to disk. Call screen_watch_latest to retrieve the most recent frame, or screen_watch_status to check progress. The master switch must be ON (screen_enable) before this works. Args: interval: Seconds between polls (default 5). max_frames: Max changed frames before auto-stop (default 10). question: Optional question to echo back for your context. Returns: Status dict with daemon PID and frame count.
Stop a running watch daemon. If no daemon is running, returns a status message indicating that. Args: question: Optional question to echo back for your context. Returns: Status dict with final frame count and stop reason.
No pagination or result limits documented for screen_list_displays. If a system has many displays, the response could grow unbounded. Should document max result count and whether pagination is supported.
Tool annotations (readOnlyHint, destructiveHint, idempotentHint) not present. screen_enable/disable are destructive (state change), screen_capture is read-only, screen_watch_* are stateful. Explicit annotations would help agents reason about side effects and retry safety.