Loading ToolBench
Server has 23 tools with mostly complete schemas and descriptions. Naming follows verb_noun convention well (get_*, create_*, update_*, delete_*). Descriptions are present but often generic (avg ~80 chars, below 194-char baseline). Parameter descriptions exist but lack detail on constraints, formats, and dependencies. Output schemas are not documented, LLMs cannot predict response structure. Error handling is minimal (basic try-catch, no recovery guidance). No tool annotations (readOnlyHint, destructiveHint). Security: API key injected via env var (good), but no audit logging or permission gates visible.
Create a new canned response.
Create a new canned response folder.
Create a new contact custom field.
Create a new agent group.
Create a ticket in Freshdesk
Delete the summary of a specific ticket.
Get list of agents from Freshdesk with pagination support.
Output schemas not documented. LLMs cannot predict response structure or plan downstream tool calls. E.g., get_tickets returns {tickets, pagination} but this is inferred from code, not declared in tool metadata.
Parameter descriptions lack constraint details. E.g., 'page' and 'per_page' in get_tickets have ranges (1-100) documented in code validation but not in parameter descriptions. LLMs cannot infer valid ranges from code.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Destructive tools like delete_ticket_summary lack explicit marking. LLMs cannot distinguish safe reads from dangerous writes without annotations.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | <=2025-11-25 | v2 |
Get a specific ticket from Freshdesk.
Get conversation/comments for a specific ticket.
Get ticket fields from Freshdesk.
Get tickets from Freshdesk with pagination support.
List canned responses from a specific folder.
List all agent groups.
List solution articles from a specific folder.
List all solution categories.
List solution folders from a specific category.
Update an existing canned response.
Update an existing canned response folder.
Update a ticket in Freshdesk.
Update a conversation/comment on a ticket.
Update the summary of a specific ticket.
View a specific canned response.
View the summary of a specific ticket.
Error handling is minimal. Code catches httpx.HTTPStatusError and generic Exception but returns bare error dicts without recovery guidance. E.g., 'Failed to fetch tickets: ...' does not tell LLM what to do next.
Generic tool descriptions. Many tools have descriptions under 20 chars or lack context on when to use them. E.g., 'Get ticket fields from Freshdesk.' does not explain what fields are returned or why an LLM would call this.