Loading ToolBench
Aussie Broadband ISP operations. Monitor broadband usage, track outages, check billing, view support tickets, and run line diagnostics. Multi-account support — pass account= to target a specific account. Diagnostic tests require ABB_DIAGNOSTICS_ENABLED=true.
10 tools with consistent naming (abb_* prefix) and complete input schemas. All tools have descriptions (10-120 chars, mostly 60-90 chars). Schemas properly typed with string/integer fields and descriptions. However, descriptions are terse and lack context about when to use each tool, dependencies between tools, or error recovery guidance. No output schemas documented. All tools are READ_ONLY, which is good for safety, but no tool annotations (readOnlyHint) are present in the code. Error handling returns generic ABBError strings without actionable recovery steps. No pagination support despite tools like abb_billing and abb_tickets that could return large result sets.
Billing transactions grouped by month: date, amount, description, type.
Add-ons (boltons) for a service: name, cost, status.
Health check: connection status, customer details, service count, diagnostics gate status.
Pending orders: order ID, status, type, description.
Outages for a service: network events, ABB outages, NBN outages (current, scheduled, resolved).
Get full details for a single service including address, plan, technology type, speed tier, and POI.
No output schemas documented. LLMs cannot plan downstream tool calls or extract specific fields without knowing response structure. All 10 tools return formatted strings without schema declarations.
Tool descriptions lack context about when to use each tool vs alternatives. E.g., abb_service vs abb_services distinction unclear. No dependency hints (e.g., 'call abb_services first to get service_id'). Descriptions average 70 chars but lack LLM-optimized guidance.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | <=2025-11-25 | v2 |
List all services (broadband, VOIP, Fetch TV). Compact one-line output with ID, type, plan, tech, speed.
Telephony usage breakdown: national, mobile, international, SMS, voicemail with call counts and costs.
Support tickets: reference, status, subject, date.
Broadband usage for a service: download, upload, remaining, billing period, percentage used.
Error handling returns generic ABBError strings without actionable recovery guidance. E.g., 'Error: <message>' tells LLM nothing about whether to retry, ask user, or try alternative tool. No error classification (retryable vs user-fixable vs fatal).
No tool annotations present. All tools are READ_ONLY but lack readOnlyHint annotation in schema. This prevents clients from optimizing caching, retry logic, or permission checks based on tool safety classification.
No pagination support. Tools like abb_billing (limit param exists but no offset/cursor), abb_tickets, abb_orders, abb_services could return unbounded lists. No total_count or next_cursor in documented output. Large result sets risk context window exhaustion.