Loading ToolBench
A lightweight and resilient multi-LLM orchestrator that routes queries to specialist roles and integrates with MCP servers for tool-based task execution
FigureOut exposes 9 tools across multiple example domains (events, flights, customer support). All tools have basic descriptions and input schemas with typed parameters. However, descriptions are minimal (avg ~40 chars, well below the 194-char baseline), parameter descriptions lack depth, output schemas are undocumented, and error handling guidance is absent. No tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications. Tools are well-named with action verbs, but lack the LLM-optimization depth expected of production systems. This is a typical community server: functional but not refined.
Create a new customer support ticket
Retrieve customer information and support history
Search events by artist name
Search events by genre
Search events by event type
Get fee breakdown for an event
Search for available flights
Get seat availability and recommendations for an event
Descriptions are too brief (avg ~40 chars vs 194-char baseline). LLMs cannot distinguish when to use get_events_by_artist vs get_events_by_genre vs get_events_by_type without richer context. Add 50-150 char descriptions explaining the search strategy and when each is preferred.
Output schemas are not documented. Callers cannot know what fields to expect (e.g., does get_events return event_id, title, date, venue, price?). Document return types for all tools to enable downstream tool chaining.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk classifications. create_support_ticket and update_support_ticket are marked WRITE but lack destructiveHint annotation. Add annotations to guide agent retry logic and safety policies.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
Update an existing support ticket
Parameter descriptions are minimal or missing. 'tier' in get_seats has no description of valid values (e.g., VIP, standard, general admission?). 'priority' in create_support_ticket lacks enum guidance. Add 30-80 char descriptions and enums for constrained parameters.
No error handling guidance. Tools do not document what happens on invalid input (e.g., invalid date format, non-existent event_id, out-of-range priority). Add recovery hints like 'If event not found, try search_events() with partial name.'