Loading ToolBench
The server defines 3 OCR tools with basic structure but significant quality gaps. All tools have descriptions (good baseline), but parameters lack key details, error handling is minimal, and output schemas are not documented. Tool names follow a decent verb_noun pattern (ocr_*), but descriptions are generic and don't explain WHEN to use each tool or WHAT each returns. Parameters like 'returnImageInfo' lack type constraints and actionable descriptions. The Zod validation in index.ts validates input schema but responses are free-text and unstructured, forcing LLMs to parse OCR results without knowing the field structure. No guidance for recovery from failures (e.g., file not found, API timeout, invalid image format). Security is reasonable (credentials via env vars), but no audit logging or permission gates.
Extract text from images using Quark Vision OCR API. Handles printed text, documents, receipts, and general text content.
Recognize handwritten text in images using Quark Vision OCR API. Best for handwritten notes, letters, and documents.
Recognize and extract table structures from images using Quark Vision OCR API. Ideal for spreadsheets, financial tables, and structured data.
Output schema not documented. Tools return OCRResult (success/error/text fields visible in code) but LLMs cannot see structured field definitions. Responses are serialized as free-text, forcing LLMs to parse unstructured output. This violates the response-shaper pattern.
Parameter 'returnImageInfo' is boolean but lacks actionable description. What does 'additional image metadata' mean? Width/height? Format? Color space? LLMs cannot reason about when to set true vs false without explicit field names and use cases.
Tool descriptions lack differentiation and 'WHEN to use' guidance. All three tools describe themselves similarly ('Recognize/Extract text using Quark Vision OCR API'). An LLM cannot determine whether to call ocr_general, ocr_handwritten, or ocr_table for a mixed document (text + handwriting + tables). Descriptions should explain: general=printed/mixed documents, handwritten=cursive/script, table=structured rows/columns.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 39 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 43 | - | v1 |
Error handling lacks recovery guidance. If encodeImage() fails (file not found, permission denied, unsupported format), the error message is raw ('error: ENOENT' or 'error: EACCES'). LLMs receive no actionable next step. Should be: 'File not found at /path/to/file. Verify the absolute path exists and is readable.' or 'Unsupported image format. Use JPEG, PNG, BMP, or TIFF.'
Parameter 'imagePath' requires absolute path but no validation enforces this. LLMs may pass relative paths, symlinks, or URLs, all of which will fail silently or with confusing errors. Schema should include a regex pattern and description should explicitly state: 'Must be an absolute POSIX path (e.g. /home/user/image.jpg on Linux) or Windows path (e.g. C:\Users\user\image.jpg). Relative paths will fail.'
No timeout or cancellation hints in descriptions. QuarkVisionClient has a 30-second timeout but this is not exposed to the agent. If the API is slow or unreachable, the agent does not know to expect a delayed response or a timeout error. Tool descriptions should state: 'This tool calls an external API (timeout 30 seconds). If the API is slow or unreachable, you will receive a timeout error. Consider retry or alternative OCR services.'
No documentation of what OCR output structure looks like. What fields does the API return? Is it plain text, JSON, structured blocks? The tools serialize to 'text' field but agents cannot plan downstream processing (e.g., extract fields, parse tables, post-process results) without knowing the structure. Add a schema or example to the tool description.