Loading ToolBench
An MCP server that provides tools to interact with a FastAPI books REST API
This server provides 5 CRUD tools for a book management API. All tools have names starting with verbs (get_, create_, update_, delete_) which is good. Descriptions exist but are minimal (10-27 characters), well below the 194-character production baseline. Input schemas are present and properly typed, but parameter descriptions are sparse. Error handling is minimal, exceptions are caught and logged to stdout, returning empty dicts/lists rather than actionable error messages. The schema and naming are adequate, but description quality and error guidance are significantly below production standards.
Create a new book.
Delete a book by its ID.
Get a book by its ID.
Get all books.
Update a book by its ID.
Descriptions are extremely minimal (10-30 chars), far below production baseline of 194 chars. LLMs lack context to select tools or understand dependencies. E.g. 'Get all books.' tells the LLM nothing about when to use this vs get_book, what fields are returned, or pagination.
No error handling guidance. All tools catch exceptions and return empty dict/list with no recovery hints. LLMs receive empty responses on failure and cannot distinguish between 'no results' and 'failed to fetch'. E.g., if the FastAPI server is unreachable, get_books() returns [] silently, indistinguishable from an empty database.
Parameter descriptions are missing entirely from the books_mcp.py tool definitions. E.g., book_id has no description in the @mcp.tool() decorator. FastMCP may infer types from function signatures, but LLMs cannot read Python type hints, they rely on description text. Parameter descriptions are 0/5 tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 40 | - | v1 |
No output schema documentation. LLMs do not know what fields are returned by each tool. get_books returns a list, but the structure is undocumented, is it {id, title, author, year, genre} or something else? This forces LLMs to guess or make trial calls.
No pagination support. get_books() returns all books with no limit, offset, or cursor parameters. If the database has 10,000 books, the response could explode the context window and degrade LLM reasoning. Production baseline: cap results at 20-50 and offer pagination.
delete_book and update_book lack confirmation/dry-run support. Irreversible operations should guard against agent mistakes. No pattern to preview changes or confirm before executing.
No validation of enum constraints. genre parameter defaults to 'Unknown' but is a free-form string. If the API enforces a fixed set of genres (e.g. fiction, non-fiction, science, history), this should be an enum to prevent invalid input.