Loading ToolBench
Two durable MCP tool servers using Resonate for automatic retries, state management, and durability across process crashes. Includes an invoice processing workflow with human-in-the-loop approval and a weather forecast tool using the National Weather Service API.
The server defines 5 tools with complete input schemas and basic descriptions. However, descriptions are minimal (averaging 90-120 chars, well below the 194-char production baseline), lack context on when/why to use each tool, and omit error handling guidance. Parameter descriptions are present but generic. Output schemas are not documented. No tool annotations (readOnlyHint/destructiveHint). The invoice tools form a coherent workflow but lack composition clarity (e.g., how does an agent know the order to call them?). Weather tool is standalone. Overall execution is functional but falls short of production quality, closer to example/demo code than a robust agent tool suite.
Approve a pending invoice for payment processing
Check the current status of an invoice
Get weather forecast for a location. Automatically retries on failure, maintains state across crashes, and handles API timeouts gracefully.
Reject a pending invoice
Submit an invoice for processing. The invoice will wait for human approval before processing payments. Automatically handles failures, retries, and maintains state across crashes.
Descriptions are below production baseline (90-120 chars vs. 194-char median) and lack actionable context. 'Approve a pending invoice for payment processing' does not tell the LLM when to call this vs. reject, what happens on approval, or whether calling it multiple times is safe. Descriptions should include WHAT, WHEN, and any side effects.
No output schemas documented. Callers (and LLMs) cannot plan downstream calls without knowing what fields are returned. E.g., what does check_invoice_status return, a status string, a full invoice object, or a structured record with line items? This forces LLMs to guess.
No error handling guidance in tool descriptions. If submit_invoice fails (e.g., invalid line items), what should the LLM do? Retry? Fix the data? Ask the user? A bare error response leaves the agent stuck.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 65 | 2026-07-28+ | v2 |
No tool annotations. approve_invoice and reject_invoice are destructive operations (state-changing), they should carry destructiveHint:true. check_invoice_status is read-only, should carry readOnlyHint:true. submit_invoice is not idempotent (calling it twice creates two invoices), should NOT carry idempotentHint. Annotations guide agent planning and safety.
Workflow composition not explicit. The invoice tools form a natural sequence (submit → approve/reject → check status), but tool descriptions do not document this order or dependencies. An agent must infer: 'If I want to approve, I must call submit first.' Add dependency hints to guide multi-step planning.
Parameter descriptions are present but minimal. 'Unique identifier for this invoice' and 'ID of the invoice to approve' do not explain format, length, or uniqueness constraints. E.g., is invoice_id a UUID, a sequential number, or a user-provided string? Can it contain spaces or special chars?
No validation or constraint documentation. The 'lines' array in submit_invoice has no min/max length constraint documented. Can an invoice have zero lines? A million lines? The amount field has no min/max. These gaps invite invalid input from LLMs.
get_forecast is vague about its output. Returns a 'weather forecast', but in what format? JSON with hourly data? A text summary? Temperature range? Precipitation probability? LLMs cannot plan without knowing the structure.