Loading ToolBench
Read a PDF file and extract its text. Works with both protected and unprotected PDFs.
The server has a single tool (read_pdf) with a complete schema and functional descriptions. The tool naming is verb-based and clear. However, the descriptions lack depth regarding when to use the tool vs. alternatives, and several critical error-handling patterns are absent. The tool returns unstructured dictionaries with variable success/error fields, making it harder for LLMs to reliably parse responses. No tool annotations (readOnlyHint, idempotentHint) are present despite being read-only. The resource definitions and prompt are well-implemented, but the core tool lacks production-grade robustness.
Read a PDF file and extract its text. Works with both protected and unprotected PDFs.
Tool description lacks guidance on when/why to use it. Missing context about supported formats, performance characteristics (e.g., 'Handles text-extractable PDFs up to X MB'), and when to use resources vs. the tool.
No structured error response format. The tool returns inconsistent dictionaries: sometimes {'success': False, 'error': '...'}, sometimes with additional fields like 'is_encrypted', 'password_required'. LLMs cannot reliably parse or handle these variations.
Error messages lack recovery guidance. E.g., 'File not found: /path/to/file' does not suggest checking the path or listing available files. Pattern requires: 'File not found at /path/to/file. Verify the path exists or use a relative path from your working directory.'
Missing tool annotations. The tool is read-only (no state modification), but no readOnlyHint or idempotentHint is declared. This prevents agents and clients from safely retrying without side effects.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 56 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 52 | - | v1 |
Output schema not formally documented. While the code returns a dict with keys like 'success', 'content', 'metadata', etc., there is no explicit JSON Schema definition for the tool's return type. This forces LLMs to infer structure from trial-and-error.
Parameter descriptions lack format/constraint details. 'password' has a description but no guidance on character encoding, length limits, or special character handling. 'pages' accepts 1-indexed integers but does not specify max length, minimum value, or maximum page count.
No input validation errors. If 'pages' contains invalid values (0, negative, or exceeding total_pages), the tool silently ignores them or fails cryptically. Should validate and return actionable errors.
Resource URI patterns use URL encoding workaround ('file_path.replace('%20', ' ')') instead of proper URI decoding. This is fragile for paths with special characters.