Loading ToolBench
A Model Context Protocol server that provides tools for querying invoice data from a PostgreSQL database
Server has 4 tools with basic schemas and descriptions, but significant gaps prevent production use. All tools lack parameter descriptions (critical for LLM reasoning). Three tools have empty input schemas with no parameters at all. Descriptions are present but generic (10-50 chars, below the 50-200 char baseline for A+ tools). No output schema documentation. No error handling guidance. No tool annotations (readOnlyHint, etc.). Tool naming is acceptable (verb_noun pattern) but descriptions don't explain WHEN to use each tool or dependencies between them.
Retrieves all invoices
Retrieves invoices whose numbers contain a provided pattern
Retrieves the number of paid invoices
Returns the total invoice paid amount
No parameter descriptions for any tool. The 'pattern' parameter in get-invoices-by-pattern-on-number lacks a description explaining what constitutes a valid pattern (regex? substring? case-sensitive?). LLMs cannot infer parameter semantics from names alone.
No output schema documentation. Tools return results as toString() strings (e.g., 'result.toString()' in InvoiceTools.java). LLMs cannot parse unstructured text output or extract fields for downstream tool calls. Structured JSON responses with typed fields are required.
Tool descriptions are too short (10-50 chars vs. baseline 50-200 chars). 'Retrieves the number of paid invoices' and 'Returns the total invoice paid amount' lack context on WHEN to use each tool, what the return value represents, or dependencies. Descriptions should answer: what does it do, when should I call it, what does it return?
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 38 | <=2025-11-25 | v2 |
No error handling guidance. Tools return isError(false) unconditionally. No recovery hints if a query fails (e.g., 'If pattern is invalid, try a simpler substring'). Errors must tell the LLM what to do next.
No tool annotations. Tools lack readOnlyHint, destructiveHint, or idempotentHint. All tools are read-only (safe to retry), but this is not declared in the schema. Clients cannot infer safety properties without annotations.