Loading ToolBench
A full-stack chatbot application with vector store memory (Qdrant), LangGraph-based agent, web search tools (Tavily, Firecrawl), LiveKit voice integration, and Mem0 memory management. Provides real-time information retrieval and conversational context.
Two tools present with basic schemas and descriptions, but significant gaps in parameter descriptions, output schema documentation, and error handling guidance. Tool names follow verb_noun convention. Descriptions exist but are generic and lack context about when to use each tool vs the other. Parameters have type definitions but lack constraint documentation (enums, ranges, format rules). No output schema documentation visible. Error handling returns raw error messages without recovery guidance. Average tool score: 52.
Scrape and extract content from a webpage using FireCrawl API.
Search the web for real-time information using Tavily API.
Output schemas not documented. Code returns structured objects (tavily: {content, error, already_stored}; firecrawl: string) but tool definitions do not declare return types. LLMs cannot plan downstream calls or know what fields to extract without documented schemas.
Parameter constraints missing. No enums, min/max, regex patterns, or format rules documented. E.g., 'query' could be 1-2000 chars; 'url' must match URL regex.
Error handling lacks LLM-actionable recovery guidance. Errors are returned as strings or objects but do not tell the agent what to do next (retry? ask user? fatal?). Per pattern:recovery-guide, errors must categorize and guide next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 46 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 42 | - | v1 |
Tool naming ambiguity. Both 'tavily_search' and 'firecrawl_search' are search tools. LLMs cannot easily disambiguate when to call which one. Names should convey the difference: search_current_web (Tavily, real-time) vs scrape_webpage (FireCrawl, static content extraction).
Descriptions too generic and lack WHEN/WHY context. Descriptions do not explain when to use tavily_search vs firecrawl_search, or what outputs to expect. Current descriptions are ~70 chars of generic action description only.
Optional parameters documented as required. tavily_search declares userId and chatId as required parameters, but code treats them as optional with fallbacks ('if userId && chatId && userId !== "" && chatId !== ""'). This mismatch forces LLMs to always provide these even when not needed, or causes failures if not provided.
No tool composition guidance. No indication that firecrawl_search results could chain into tavily_search, or vice versa. Per pattern:tool-chain, tool outputs should include IDs/references needed for downstream calls.