Loading ToolBench
MCP server exposing PostgreSQL as agent-callable tools
Three tools with complete schemas and descriptions, but significant gaps in parameter documentation and output structure. inspect_schema and query have good descriptions (150+ chars) and proper input schemas with types. search_books has a well-documented interface with all params described. However, NO tool documents its output schema, LLMs cannot plan downstream calls or extract structured data. Parameters lack constraints (e.g., min_rating range 0.0-5.0 stated in description but not in schema; limit max 50 stated in description but not enforced in schema). Error handling is minimal, tools return plain strings rather than structured error objects with recovery guidance. No tool declares idempotency or safety hints. Naming is clear and verb-forward (inspect_, query, search_), following baseline conventions.
Inspect the database schema. Without arguments, lists all tables and their row counts. With a table_name, returns columns, types, nullability, defaults, and constraints for that table.
Execute a read-only SQL query against the database. Safety: the query runs inside a READ ONLY transaction with a statement timeout. DDL and DML keywords are rejected. Results are capped at the configured row limit. Args: sql: A SELECT query. Must not contain INSERT, UPDATE, DELETE, DROP, ALTER, CREATE, TRUNCATE, GRANT, REVOKE, COPY, or EXECUTE. Use $1, $2, ... for parameter placeholders. params: Optional list of parameter values matching the $N placeholders.
Search for books by title, author name, or genre. Uses case-insensitive pattern matching across title, author name, and genre name. Results can be filtered by minimum rating and stock availability. Args: search_term: Text to search for in book title, author name, or genre. min_rating: Optional minimum rating filter (0.0 to 5.0). in_stock: Optional filter for stock availability. limit: Maximum number of results (default 10, max 50).
No output schemas documented. All three tools return plain strings; LLMs cannot parse structured results or plan chained calls. inspect_schema returns formatted text; query returns tab-separated rows; search_books returns formatted text. Without documented output structure, agents waste tokens parsing unstructured output.
Parameter constraints stated in descriptions but not enforced in schemas. search_books 'limit' described as 'max 50' but schema has no maximum constraint. min_rating described as '0.0 to 5.0' but no schema bounds. Constraints must be in JSON Schema (minimum, maximum, enum) so LLMs and validators can enforce them.
Error handling returns plain strings with no recovery guidance. query() returns 'Error: only SELECT queries are allowed...' but does not suggest what to do next. No error categorization (retryable vs fatal). Errors should guide LLM to next action: 'Invalid query. Try: search_books() to find available tables, or inspect_schema() to see column names.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
No tool annotations (readOnlyHint, idempotentHint). All three tools are read-only and idempotent, but this is not declared in the tool definition. Agents cannot infer safety properties and may incorrectly treat them as stateful or destructive.
query() tool lacks pagination support. Results capped at ROW_LIMIT (100-500) but no offset/limit parameters or next_cursor returned. Large result sets truncated silently; agents cannot fetch remaining rows. Should accept limit and offset params and return total_count.