Loading ToolBench
A TypeScript support agent using REST, MCP, local embeddings, RAG, and LanceDB for ticket management and document search
Server has 4 tools with basic schemas and descriptions. All tools have names starting with action verbs (get_, create_, search_) and descriptions present. However, descriptions are minimal (10-50 chars), parameter descriptions are missing or trivial, and output schemas are completely undocumented. Error handling exists but lacks recovery guidance. Tool annotations present on create_ticket but missing context on what they mean. No pagination support despite search_documents potentially returning large result sets. Schemas use Zod validation but lack formal JSON Schema documentation in descriptions.
Create a new support ticket with the provided title
Get one ticket using its numeric ID
Get all tickets from the REST API
Search support documents by meaning, optionally within a category
Output schemas completely undocumented. All tools return JSON text wrapped in content array, but LLMs cannot infer field structure, types, or what to extract for downstream calls. No documentation of ticket object shape, search result format, or error response structure.
Parameter descriptions missing or trivial. 'id' param in get_ticket has description but lacks context on valid range or format. search_documents 'query' and 'category' descriptions are present but minimal. No guidance on minimum/maximum values, format constraints, or when to use each parameter.
Tool descriptions too short (10-50 chars). Baseline for A+ tools is 194 chars average. Descriptions lack WHEN to use, WHAT it returns, and dependencies. E.g., 'Get all tickets from the REST API' does not explain when to call this vs get_ticket, or what structure is returned.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
No pagination support. search_documents and get_tickets may return large result sets but lack limit, offset, page, or cursor parameters. No documentation of result count limits. LLMs cannot control response size, risking context window exhaustion.
Error handling lacks recovery guidance. get_ticket returns isError: true with raw JSON, but does not tell LLM what to do next (e.g., 'Try search_tickets() with a partial title'). Error messages are not actionable for agent self-correction.