Loading ToolBench
Back-office data for a small business. Amounts are USD. Use overdue_invoices before drafting any payment reminder.
BizDesk has clear, verb-prefixed tool names and basic parameter schemas with types and descriptions. However, output schemas are undocumented (no return type specs visible), error handling is minimal, and parameter descriptions lack detail on constraints and formats. Tool descriptions are adequate (50 - 150 chars) but lack context on when to use each tool or what happens on failure. The add_note tool has no error guidance for missing customers. Overall: functional but below production baseline.
Attach a timestamped note to a customer record.
List customers. status: all | active | churned.
Invoices past their due date, most overdue first.
Paid revenue by month for a year, plus totals and top customer.
Output schemas not documented. Tools return dicts/lists but LLMs cannot see field names, types, or structure. Forces LLMs to guess what fields exist and risk extraction errors.
Parameter descriptions lack format/constraint details. 'status' in list_customers says 'all | active | churned' but should be an enum. 'min_days_overdue' has no range (1 - 365?). 'year' in revenue_summary has no bounds.
add_note returns a string message ('No customer named X found') instead of structured error. LLMs cannot distinguish success from failure or know what to do next. Should return {success: bool, message: str, customer_id?: int}.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
Tool descriptions lack context on when to use each tool. 'List customers' vs 'revenue_summary', when should an LLM pick one over the other? No guidance on prerequisites or dependencies.
No pagination support. list_customers and overdue_invoices return all results. If a business has 10k customers, response bloats context window. Should accept limit/offset and return total_count.