Loading ToolBench
Pergamos has clear, action-verb tool names (list_, search_, get_, index_, search_) and all tools include descriptions. Input schemas are present with type definitions and parameter descriptions. However, output schemas are not formally documented, tools return JSON strings without declared response structures. Error handling returns JSON with error fields but lacks recovery guidance. Parameter validation is inline but not schema-enforced (e.g., limit bounds checked in code, not in schema). Tool descriptions are adequate (100-200 chars) but could be more LLM-optimized with dependency hints and use-case clarity.
Return detailed metadata and available format links for one Calibre book.
Download a book file, extract text, split it into chunks, and index the content for semantic search.
List the libraries exposed by the configured Calibre Content Server.
Search the indexed text for relevant chunks within one or more Calibre books.
Search Calibre books by full text and return metadata plus available formats.
Output schemas not formally documented. Tools return JSON strings without declared response structures. LLMs cannot plan downstream calls or extract fields reliably.
Parameter validation constraints (limit 1-100, offset ≥0, k>0) are enforced in code, not declared in JSON Schema. LLMs cannot see bounds before calling and may pass invalid values.
Error responses lack recovery guidance. Errors return JSON with 'error' field but do not suggest next steps (e.g., 'Try search_books() first' or 'Check CALIBRE_SERVER_URL').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | 2026-07-28+ | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). index_book_content is destructive/stateful but not marked; search tools are read-only but not annotated.
search_books and search_book_content lack pagination documentation in descriptions. Agents may not know results are capped or how to fetch more.