Loading ToolBench
Deterministic book-to-markdown pipeline for LLM consumption. Supports MCP server mode for integration with Claude and other LLM clients.
librarian-cli defines 7 tools with a mix of complete and incomplete documentation. All tools have descriptions (10 - 140 chars, averaging ~90 chars), which is on the short side against the 194-char baseline but acceptable for discovery. Input schemas are present and use proper JSON Schema with type declarations. However, OUTPUT schemas are not documented, there is no visibility into what fields are returned by any tool, breaking the chain of composition and making it impossible for an agent to extract the right data for downstream calls. Error handling is minimal; no recovery guidance is provided. Naming is generally clear and verb-based (list_, get_, search_, verify_, remove_, ingest_). Parameters lack detailed constraints (enum values, ranges, format specifications) and descriptions are terse, most are 1-2 sentences vs the 72-char baseline. Tool risk annotations (READ_ONLY, WRITE, DESTRUCTIVE) are declared but not reflected in the schema as idempotentHint/destructiveHint. Overall: solid foundation with clear intent, but incomplete schema documentation and missing error-recovery patterns.
Get detailed metadata and quality metrics for a specific book
Get chapter text from a book. Can select specific chapters by number range or by token budget.
Add a new book to the library by processing a source file (ebook, PDF, text, etc.)
List all books in the library or get detailed chapter information for a specific book
Delete a book from the library
Full-text search across all books in the library
Verify whether a quote exists in the library books and assess accuracy
Output schemas not documented. No visibility into what fields tools return (e.g., does list_books return book_ids, titles, authors, word_counts?). Agents cannot extract data for downstream calls and must infer structure from usage.
No error handling or recovery guidance. Tools offer no messages that guide the LLM on what to do if a call fails (e.g., 'book not found, try search() first', 'invalid chapter range, valid range is 1-42'). Agents have no recovery path.
Parameter descriptions lack constraint specification. 'chapter range specification (e.g. '1', '1-3', '1,3,5')' implies format but does not enforce it, no regex, no enum, no min/max chapter count. Agents must guess valid ranges and risk failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | 2026-07-28+ | v2 |
Tool risk annotations (READ_ONLY, WRITE, DESTRUCTIVE) declared in comments but not exposed in schema. Modern MCP spec requires idempotentHint, readOnlyHint, and destructiveHint in tool definitions so clients can make informed execution decisions. Missing.
remove_book description does not explicitly state it is irreversible and destructive. Agents need clear warning that this call deletes data. No dry-run or confirmation step offered, a critical design gap for destructive operations.
ingest tool uses a file path parameter with no path traversal or injection defense documented. If an agent is compromised or prompt-injected, a path like '../../../sensitive_file' could be passed. No input validation or sanitization guidance.
Parameter descriptions use example values in parentheses (e.g., '(e.g. '1', '1-3', '1,3,5')') instead of formal constraints. LLMs tend to reuse example values literally rather than adapt them, causing failures when the examples don't match the user's actual data.