Loading ToolBench
BookMCP defines 5 read-only tools with consistent naming (verb_noun pattern: book_search, book_get_*, lesson_list) and complete input schemas using JSON Schema with type and description fields. All tools have descriptions (34-89 chars, within baseline range). However, output schemas are not documented in the source code, critical for LLM planning. Parameter descriptions are minimal (10-30 chars) and lack format/constraint details. No error handling guidance, no pagination metadata (total_count, next_cursor), and no tool annotations (readOnlyHint). Tools are well-composed (single responsibility) but lack the depth expected of production-grade tools.
Fetch one extracted chunk by book ID and chunk ID
List ingested books with pagination support
Fetch one extracted page by book ID and one-based page number
Search indexed book chunks by keyword query, optionally filtered by book ID
List saved lessons with optional filtering by book ID and pagination
Output schemas not documented. LLMs cannot plan downstream calls or extract required fields (e.g., does book_search return book_id, title, chunk_id for chaining?). Baseline: 100% of A+ tools document return types.
Parameter descriptions are too brief (10-30 chars). Lack format/constraint details. E.g., 'top_k' has no range (1-100? 1-1000?), 'page_number' lacks 'one-based' clarity in description. Baseline: avg param annotation 72 chars.
Pagination tools (book_get_library_index, lesson_list) accept offset/limit but do not document return of total_count or next_cursor. LLMs cannot determine if more results exist or plan multi-page fetches. Baseline: paginated results must include count/cursor.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | <=2025-11-25 | v2 |
No tool annotations (readOnlyHint). All tools are read-only but lack explicit annotation. Agents cannot distinguish safe tools from destructive ones without parsing descriptions.
No error handling guidance. If book_id or chunk_id is invalid, what error is returned? How should the LLM recover? Baseline: errors must guide recovery (e.g., 'Try search_books() first').