Loading ToolBench
Expense Tracker has 12 tools with basic schemas and descriptions, but significant gaps prevent higher scores. All tools have descriptions (10-100 chars, mostly adequate), and input schemas are present with type definitions. However, output schemas are completely undocumented, LLMs cannot predict what fields to expect from responses. Parameter descriptions are minimal (5-20 chars), lacking format guidance, constraints, or dependency hints. No error handling guidance; tools return generic {'status': 'ok'} or {'status': 'error'} without recovery hints. No pagination support despite list_* tools. Tool naming is verb-first and clear (add_, delete_, edit_, list_, summarize_), which is a strength. Security: no secrets exposed as params (good), but no permission gates or audit logging. Composition is reasonable, tools are single-purpose, but output lacks chaining IDs (e.g., list_expenses returns id but edit_expense needs expense_id; naming is consistent but not all tools return the IDs downstream tools need).
Add a new credit (income) entry to the database.
Add a new expense entry to the database.
Delete an expense entry by its ID.
Edit an existing credit (income) entry by its ID. Only provided fields will be updated.
Edit an existing expense entry by its ID. Only provided fields will be updated.
List all categories from the categories.json file.
List credit (income) entries within an inclusive date range.
Output schemas completely undocumented. LLMs cannot predict response structure (fields, types, nesting). list_expenses returns dict list but no schema; summarize returns category+total_amount but not documented. Agents must guess or fail.
Parameter descriptions are minimal (5-20 chars) and lack format/constraint guidance. 'Date of the expense' does not specify format (YYYY-MM-DD? ISO 8601?). 'Amount' lacks range (0-1M? negative allowed?). LLMs cannot validate inputs.
No error handling guidance. Tools return {'status': 'error', 'message': '...'} but do not tell LLM what to do next. 'No fields to update' error in edit_expense gives no recovery path. Agents cannot self-correct.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
List expense entries within an inclusive date range.
List all custom salary summaries saved as text.
Save a custom salary summary as plain text for future reference. The summary_text should be a detailed description (e.g., generated by Claude or another AI).
Summarize expenses by category within an inclusive date range.
Summarize credits by category within an inclusive date range.
No pagination support. list_expenses, list_credits, list_categories, list_salary_summaries accept no limit/offset/page params. Large result sets will blow context window. Baseline pattern requires pagination.
Destructive operations (delete_expense) lack confirmation/dry-run. Agents can permanently delete records without a safety gate. No idempotent hints or confirmation request pattern.