Loading ToolBench
Local OCR and on-device screen reading for MCP clients (Claude, Cursor): read any window, locate elements to click, assert what's on screen. Apple Vision, no API keys, nothing uploaded.
Strong tool definitions with comprehensive descriptions (avg 180 chars, well above 34-char baseline) and detailed parameter documentation. All 13 tools have clear, action-oriented names (ocr_image, detect_faces, capture_screen, etc.) following verb_noun convention. Input schemas are present and typed for all tools. However, output schemas are not formally documented in the tool definitions, responses are described in prose but lack structured JSON Schema declarations. Error handling guidance is minimal; tools do not indicate what to do on failure or how to recover. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all being read-only operations. Parameter descriptions are strong but some lack explicit constraints (e.g., fuzzyThreshold 0 - 1 range is stated but not enforced via schema minima/maxima).
Full pipeline returning structured JSON suitable for reconstructing the document into Markdown, HTML, DOCX, or any other format. Includes: paragraphs[] (reading-order text grouped into paragraphs and lines), textBlocks[] (every recognized block with bbox/lineId/paragraphId), faces, barcodes, rectangles (parallel detection sections). PDFs are split per-page; coordinates are page-local 0–1. Accepts start_page / max_pages range options.
Deterministic pass/fail that text is present on or absent from the screen, computed here. A verdict costs ~240 tokens against ~6,900 for the screenshot it replaces.
Capture a display, a window (even one behind others), an app's frontmost window, or a screen region. Writes a PNG to disk and returns its path, pixel size and the screen rect it covers — never image bytes.
Classify image content into 1000+ categories with confidence scores.
Read QR codes, EAN, UPC, Code128, PDF417, Aztec, DataMatrix, and more.
Output schemas not formally documented. Tool descriptions explain response structure in prose (e.g., 'returns JSON with pages, paragraphs, textBlocks') but lack JSON Schema definitions. LLMs cannot reliably parse unstructured prose responses or plan downstream tool chains without explicit output schemas.
No tool annotations despite all tools being read-only. Tools should declare readOnlyHint=true to signal to agents that these operations are safe to retry and do not modify state. This is especially important for screen capture and OCR tools that agents may call repeatedly.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2026-07-28+ | v2 |
Locate the quadrilateral of a document in a photo — receipts, paper forms, IDs shot at an angle — returning the four corner points and a confidence score. Useful for crop / deskew hints before OCR.
Detect human faces and return their count and bounding box positions.
Find an element by its visible text and get clickPoint {x, y} in global screen points, ready to hand to an input driver. Matching normalizes unicode and whitespace, then tries exact → substring → fuzzy, reporting near misses so an OCR misread is distinguishable from absent text. This server locates; it does not click.
On-screen windows front-to-back with their global screen-point bounds, app name and pid. Use it to pick a target for the tools below.
Extract text from images or PDFs using Apple's Vision OCR engine. Supported formats: jpg, jpeg, png, heic, heif, tiff, bmp, pdf. Modes: 'text' (single plain-text string, default) or 'blocks' (JSON with pages, paragraphs, textBlocks). PDF page range: start_page / max_pages (both 1-based) restrict OCR to a slice of a multi-page PDF.
Capture and OCR in one step — read what an app is showing right now, including a window that is not in front.
The whole layout as JSON: every element's measured box, role, label and state from the accessibility tree, optionally colours and fonts, plus the visible text the tree does not account for (which doubles as an accessibility finding). Read budget.capped before trusting completeness; axTextCoverage is null when the walk was capped, deliberately.
macOS version, permission state including screenLocked, displays, and the Vision feature flags available on this machine.
Numeric parameter constraints not enforced in schema. fuzzyThreshold (0 - 1), minConfidence (0 - 1), and page ranges lack minValue/maxValue in JSON Schema. LLMs may pass out-of-range values; validation happens at runtime, not at schema parse time.
Error handling guidance absent. Tool descriptions do not explain what happens on failure (e.g., 'If OCR confidence is below minConfidence, no text is returned' or 'If window is not found, returns error code 404'). Agents cannot plan recovery without knowing failure modes.
Mutually exclusive parameters not documented. capture_screen accepts display, window, or region, but descriptions do not state which combinations are valid or what happens if multiple are provided. LLMs may pass conflicting parameters.