Loading ToolBench
A Python-based agent system with multiple AI-powered applications including library management, startup research, and therapist chatbot
This library management MCP server has fundamental definition quality issues. While all 8 tools have schemas and descriptions present, most descriptions are generic and lack the depth needed for LLM tool selection. Parameter naming is inconsistent (some tools use generic names like 'member' instead of 'member_id'), and several tools have ambiguous or incomplete descriptions that do not clearly state WHAT the tool does, WHEN to use it, or what it returns. The server splits tools across multiple files (local_server.py, mcp_service.py, library.py, assistant.py) making the actual tool definitions scattered and some inferred rather than directly visible. Error handling is absent, no recovery guidance, no invalid input examples, no actionable error messages. Output schemas are documented via Pydantic models but not explicitly described in tool docstrings, forcing LLMs to infer expected fields.
Adds a new book and its copies to the library.
Borrows a book for a member. Return detailed loan data information including load id and copy id
Returns a borrowed book copy.
Gets all loans for a specific member.
Get the user's member_id information
Returns a borrowed book copy.
Searches for books by title or author.
Parameter naming inconsistency: search_member accepts 'member' parameter but spec lists 'name'; implementation ignores input and returns all members. Tool behavior does not match description.
get_book description is identical to return_book and incorrect, says 'Returns a borrowed book copy' but actually retrieves book metadata by ID. Tool description is misleading for LLM selection.
No error handling or recovery guidance in any tool. Descriptions lack actionable error messages (e.g., what if book is unavailable? member not found? copy already borrowed?). LLM cannot handle failures intelligently.
Tool descriptions lack output schema documentation. Pydantic models (Loan, Book, Member) are defined in code but not described in tool descriptions. LLM must infer what fields to expect.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 56 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 42 | - | v1 |
Gets library member by name.
Pagination not addressed. search_books returns list[Book] with no limit, offset, or total count. Returning hundreds of results wastes context tokens and degrades LLM reasoning.
Generic parameter descriptions ('Book identifier', 'Member identifier', 'Search query') do not provide format constraints, ranges, or examples of valid input. Baseline for A+ tools is comprehensive parameter descriptions (72 chars avg, 100% of params described).
Tool definitions scattered across 4 files (local_server.py, mcp_service.py, library.py, assistant.py). Some tools are inferred from partial code visibility (e.g., my_member_info, get_member_loans). Not all tool registrations are directly visible in source, violating evidence-based scoring.
No write-safety differentiation. Tools that modify state (add_book, borrow_book, return_book) lack tool annotations (destructiveHint/idempotentHint). Agent cannot determine which calls are safe to retry.
No confirmation or dry-run for destructive operations. borrow_book and return_book modify state irreversibly but have no confirmation step. Agents cannot plan reversible operations.