Loading ToolBench
18 tools with mostly complete schemas and descriptions. Naming follows verb_noun convention well (create-, list-, get-, delete-). Descriptions are present and actionable (avg ~150 chars), exceeding the 10-char minimum. However, several critical gaps: (1) nested object schemas lack full type definitions (lineItems in create-invoice, create-bank-transaction use object properties without explicit type constraints); (2) parameter descriptions are inconsistent in depth, some lack format guidance (e.g., date fields say 'YYYY-MM-DD format' but no validation hint); (3) no output schemas documented for any tool; (4) error handling is basic (generic error text, no recovery guidance); (5) tool annotations (readOnlyHint, destructiveHint) absent despite clear risk levels (WRITE, DESTRUCTIVE, READ_ONLY). Composition is sound, tools are single-responsibility and chainable (e.g., list-contacts → create-invoice). Baseline: 549 production tools average 18-char names (this server: 15-27 chars, good); 94% have docstrings (this: 100%, good); 100% A+ tools have param descriptions (this: ~85%, gap).
Create a bank transaction in Xero. When a bank transaction is created, a deep link to the bank transaction in Xero is returned. This deep link can be used to view the bank transaction in Xero directly. This link should be displayed to the user.
Create a contact in Xero. When a contact is created, a deep link to the contact in Xero is returned. This deep link can be used to view the contact in Xero directly. This link should be displayed to the user.
Create a credit note in Xero. When a credit note is created, a deep link to the credit note in Xero is returned. This deep link can be used to view the credit note in Xero directly. This link should be displayed to the user.
Create an invoice in Xero. When an invoice is created, a deep link to the invoice in Xero is returned. This deep link can be used to view the invoice in Xero directly. This link should be displayed to the user.
Create an item in Xero.
Nested object schemas (lineItems in create-invoice, create-bank-transaction, create-credit-note, create-quote) lack explicit type constraints. Properties defined as objects with properties but no minProperties, additionalProperties, or required field validation. LLMs may pass incomplete or malformed line items.
No output schemas documented for any of the 18 tools. Callers cannot predict response structure, forcing LLMs to infer fields. Downstream tools expecting specific IDs (e.g., contactID from create-contact) have no formal contract.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
Create a manual journal in Xero. Retrieve a list of account codes in Xero to use for the journal lines. Journal lines must contain at least two individual journal lines with account codes, use basic accounting account types pairing when not specified, and make sure journal line pairs have credit and debit balanced.
Create a payment against an invoice in Xero. This tool records a payment transaction against an invoice. You'll need to provide the invoice ID, account ID to make the payment from, and the amount. The amount must be positive and should not exceed the remaining amount due on the invoice. A payment can only be created for an invoice that is status AUTHORIZED A payment can only be created for an invoice that is not fully paid When a payment is created, a deep link to the payment in Xero is returned. This deep link can be used to view the payment in Xero directly. This link should be displayed to the user.
Create a quote in Xero. When a quote is created, a deep link to the quote in Xero is returned. This deep link can be used to view the quote in Xero directly. This link should be displayed to the user.
Create a new payroll timesheet in Xero. This allows you to specify details such as the employee ID, payroll calendar ID, start and end dates, and timesheet lines.
Create a tracking category in Xero.
Create tracking options for a tracking category in Xero.
Delete an existing payroll timesheet in Xero by its ID.
Retrieve a single payroll timesheet from Xero by its ID. This provides details such as the timesheet ID, employee ID, start and end dates, total hours, and the last updated date.
Lists all accounts in Xero. Use this tool to get the account codes and names to be used when creating invoices in Xero
Lists the aged payables in Xero. This shows aged payables for a certain contact up to a report date.
Lists the aged receivables in Xero. This shows aged receivables for a certain contact up to a report date.
List all bank transactions in Xero. Ask the user if they want to see bank transactions for a specific bank account, or to see all bank transactions before running. Ask the user if they want the next page of quotes after running this tool if 10 bank transactions are returned. If they do, call this tool again with the next page number and the bank account if one was provided in the provided in the previous call.
List all contact groups in Xero. You can optionally specify a contact group ID to retrieve details for that specific group, including its contacts.
Tool annotations (readOnlyHint, destructiveHint, idempotentHint) are absent despite clear risk levels declared (WRITE, DESTRUCTIVE, READ_ONLY). LLMs cannot distinguish safe reads from destructive writes without explicit hints.
Error handling is generic. Tools return 'Error creating X: {error}' with no recovery guidance. LLMs cannot determine if errors are retryable, user-fixable, or fatal. No actionable error messages (e.g., 'Invalid contact ID. Call list-contacts to find valid IDs.').
Date parameters (date, startDate, endDate) lack explicit format validation or constraints. Descriptions say 'YYYY-MM-DD format' but no regex pattern or minLength/maxLength. LLMs may pass invalid dates like '2024-13-45'.