Loading ToolBench
Dated deliverables against a quote or work order: due dates, sign-off, what is late, milestones.
Strong tool naming (verb_noun pattern consistently applied across all 12 tools). All tools have descriptions (avg 180 chars, within baseline 194). All parameters have type definitions and descriptions. Input schemas are complete with proper JSON Schema constraints (enums, maxLength, patterns). Output schemas are NOT documented in the source code provided, this is a significant gap. Error handling is present but generic (no recovery guidance visible). Tool composition is excellent: clear single responsibilities, proper chaining via IDs (schedule, deliverable params). No security issues detected (no secrets in params). Descriptions are LLM-optimized and include examples (e.g., 'DS-2026-0001', 'WO-2026-0001'). However, example values in descriptions violate best practice, LLMs may reuse them literally.
Add a deliverable to a schedule: a description, a due date, and its value in minor units if it is separately priced. The deliverable is created in planned status.
Delete a deliverable from a schedule. The deliverable is removed from the record entirely. A deliverable that has to be redone is a new deliverable with its own due date, so the record keeps both the miss and the fix.
Move a deliverable through its status: from planned to in progress or delivered, from in progress to delivered, or from delivered to accepted. Every move is dated. A deliverable is not accepted straight from planned or in progress: acceptance is the client's answer to a handover.
Update a deliverable's description, due date, value or tax rate. The deliverable's id and its status history are fixed.
Open a delivery schedule against a quote, work order or change order and return its DS-YYYY-NNNN number: the reference, its own date, the client, a title and the currency. Free tier: 3 open.
Output schemas not documented in source code. LLMs cannot plan downstream calls or extract fields without knowing response structure.
Example values embedded in parameter descriptions (e.g., 'DS-2026-0001', 'WO-2026-0001'). LLMs tend to reuse example values literally in real calls, causing failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 65 | 2026-07-28+ | v2 |
Delete a delivery schedule. Only empty schedules (all deliverables accepted or deleted) can be deleted. Deleting a schedule does not delete its deliverables from the record; it only removes the schedule itself.
Return a delivery schedule as a plain-text document: the reference, client, title, currency, all deliverables with their due dates, current status, delivered and accepted dates, acceptance notes, lateness, value and tax rate, late deliverables summary, accepted value, and the date the schedule was created and last updated. Pro tier only.
Read one delivery schedule in full: its reference, client, title, currency, completion status, all deliverables with their due dates, current status, delivered and accepted dates, acceptance notes, lateness, value and tax rate, late deliverables summary, accepted value, and the date the schedule was created and last updated.
List all delivery schedules, with their reference, client, title, currency, completion status, counts of deliverables by status, total value, and the next due date. Free tier note included.
Update a delivery schedule's client, title or note. The reference and its date are fixed once the schedule is created.
List every deliverable that is late as at a date you pass, across all schedules: its schedule id, reference, client, deliverable id, description, due date, current status, lateness state, days late, delivered date if any, accepted date if any, currency, and value. Sorted by how late it is: the worst first. Free tier: this is free on every tier.
Return the accepted deliverables on a schedule as two payloads ready to pass to invoice_create and quote_create: one for each accepted deliverable, with its description, due date, delivered date, value in the right units for each tool, and VAT. The two tools take the same milestone in different units, so this builder is the only place the scale is applied. Pro tier only.
No error recovery guidance in tool descriptions. When a tool fails (e.g., 'schedule not found'), LLMs have no guidance on what to try next.
Destructive operations (delivery_schedule_delete, deliverable_delete) lack confirmation or dry-run support. Agents may accidentally delete data.
Tool descriptions mention tier restrictions ('Free tier: 3 open', 'Pro tier only') but no mechanism visible to enforce or communicate tier limits to the agent.