Loading ToolBench
Single tool 'search' with minimal but present schema and description. Tool is explicitly registered in setupToolHandlers() with name, description, and input schema. However, multiple critical gaps: (1) Description is only 32 characters, below the 10 - 1024 baseline and well below the 194-char average for A+ tools. (2) No output schema documented anywhere, LLMs cannot plan downstream operations or know what fields to expect from search results. (3) Parameter description ('The search query') is generic and lacks guidance on format, length, or constraints. (4) No error handling guidance, axios errors are caught but only return raw API error text without recovery hints. (5) No mention of rate limits, pagination, or result caps despite calling an external API. (6) Tool performs a stateful action (web search with side effects on API quota) but description does not acknowledge this. (7) No validation of query length or format before API call, LLMs could pass excessively long or invalid queries.
Search the web using Perplexity AI
Tool description is only 32 characters, far below the 10 - 1024 baseline. Must explain WHAT it does, WHEN to use it, and prerequisites. Current: 'Search the web using Perplexity AI' lacks context for LLM tool selection.
Output schema is completely undocumented. LLMs cannot infer what fields search() returns. Response is hard-coded to return {type: 'text', text: <api_response>}, but no schema tells the LLM what to expect or how to chain calls.
Parameter 'query' lacks constraints and validation guidance. No mention of minimum/maximum length, character restrictions, or format requirements. LLMs may pass oversized or malformed queries that fail at the API.
Error handling returns raw API error text without recovery guidance. Example: 'Perplexity API error: <error_message>' gives the LLM no actionable next step. Should categorize as retryable vs fatal and suggest alternatives.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 43 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 43 | - | v1 |
No pagination or result limiting. External API calls may return large responses that blow context windows. Code has no limit, offset, or cursor handling. Should cap results at 20 - 50 items and offer pagination parameters.
No rate limiting or timeout handling. An LLM in a retry loop could overwhelm the Perplexity API. Missing explicit timeout configuration and rate-limit guidance in tool description.