Loading ToolBench
Open files that no longer open. Runs real desktop applications inside a disposable Solari VM and returns a readable PDF, text and proof it rendered.
Four tools with complete JSON schemas and solid descriptions (avg 180 chars). Tool names follow verb_noun pattern (identify_, read_, check_). Schemas properly typed with required fields. However, parameter descriptions lack depth, many omit constraints, ranges, or usage guidance. Error handling is present but generic. No tool annotations (readOnlyHint, idempotentHint). Output schemas not documented. Missing guidance on when to call each tool vs. alternatives.
Verify that Openable is configured correctly: that the API key works and that a converter template is set. Costs nothing and starts no virtual machine. Call this first if a rescue fails, or before the first use.
Identify what a file actually is from its bytes rather than its extension, without opening it. Instant and free: no virtual machine is started. Use this first to decide whether a file is worth recovering.
Read a file that cannot be parsed directly: legacy Office documents, CorelDRAW drawings, WordPerfect files, corrupt PDFs, or anything with an unknown format. The file is opened inside a disposable hardware-isolated virtual machine using real desktop applications, and the text and page images are returned. The machine is destroyed afterwards, so untrusted files never touch this host. Takes roughly 20 seconds.
Read several unopenable files in one call. They are all opened on a single disposable virtual machine, so the boot cost is paid once instead of once per file. Boot is about two thirds of a single rescue, so five files in one call take far less than five separate calls. Prefer this whenever you have more than one file. The machine is destroyed afterwards.
Parameter descriptions lack constraint documentation. 'max_pages' says 'How many page images to render, 1 to 50' but does not state this as a formal constraint (min/max). 'paths' array lacks explicit max length enforcement in description (only in comment). LLMs cannot parse comments.
No output schema documented. Tools return ToolResult with content blocks, but LLMs have no visibility into response structure. Clients cannot plan downstream calls or extract fields reliably.
No tool annotations (readOnlyHint, idempotentHint, destructiveHint). All four tools are read-only and idempotent, but this is not declared. LLMs cannot infer safety properties and may over-retry or under-cache results.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Error handling is generic. Code shows 'fail(id, code, message)' but no guidance on recovery. E.g., if a file is not found, the error should suggest 'verify the path' or offer alternatives. No error classification (retryable vs. user-fixable).
Tool composition guidance missing. No description explains when to call identify_file first vs. read_unopenable_file directly, or when to batch with read_unopenable_files. LLMs must infer from descriptions alone.