Loading ToolBench
Server has 4 tools with basic schemas and descriptions, but significant gaps limit production readiness. All tools have descriptions (10-100 chars, mostly adequate) and input schemas with types. However, parameter descriptions are sparse or generic, output schemas are undocumented, and error handling is absent. No tool annotations (readOnlyHint/destructiveHint). Tool naming is reasonable (verb_noun pattern) but lacks clarity on side effects. The RAG tools lack pagination despite potentially returning large result sets. Web search tools accept external API keys implicitly (Tavily client initialized without visible credential injection). Overall: functional but below production baseline.
Add information of me to the vectorstore for better RAG.
Crawl a URL using Tavily API.
Perform retrieval-augmented generation (RAG) of me from vectorstore.
Perform a web search using Tavily API.
Output schemas undocumented. retrieve_augmented_generation returns List[Chunk], add_information_to_vectorstore returns None, search_web and crawl_url return dict[str, str], but LLMs cannot infer field structure or plan downstream calls without explicit schema documentation.
Parameter descriptions are generic or missing context. 'metadata' param in retrieve_augmented_generation and add_information_to_vectorstore lacks guidance on valid keys, structure, or filtering semantics. 'time_range' in search_web has enum but no explanation of what each value means (e.g., does 'day' mean last 24h or calendar day?).
No error handling or recovery guidance. Tools do not document failure modes (e.g., vectorstore connection failure, Tavily API rate limit, invalid query). LLMs receive raw exceptions with no actionable next steps.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | <=2025-11-25 | v2 |
No tool annotations. add_information_to_vectorstore is destructive (WRITE risk) but lacks destructiveHint annotation. retrieve_augmented_generation is read-only but lacks readOnlyHint. Agents cannot infer safety properties without explicit hints.
Credential injection not visible. TavilyClient() in search_web and crawl_url is initialized without explicit API key parameter, assumes environment variable or global config. If API key is in .env, it's not server-side secret injection per pattern:secret-injection. Credentials must never be agent-visible.