Loading ToolBench
A retrieval-augmented generation (RAG) system that answers staff questions from PlanSmart's internal documentation and the open web. Provides grounded question-answering with citation tracking, web research capabilities, and vector-based semantic search over a knowledge base.
Single tool 'scrape_url' has a clear, action-verb name and a reasonable description (89 chars, within 10-1024 range). However, the input schema is minimal: only a 'url' parameter with basic string type and description. No output schema is documented anywhere in the visible code. The tool lacks error handling guidance, parameter constraints (e.g., URL format validation, timeout behavior), and does not address security concerns (e.g., what happens with malicious URLs, rate limiting, or credential leakage in responses). The description does not explain when to use this tool vs. alternatives, what format the output takes, or how to handle failures. No tool annotations (readOnlyHint, etc.) are present. This is a bare-minimum tool definition that would not pass production code review.
Fetch and clean the text content of a public web page for use as external evidence in grounded answers.
No output schema documented. LLMs cannot plan downstream operations or extract structured data from the response.
Input schema lacks constraints. 'url' parameter has no format validation, length limits, or enum of allowed domains. LLMs may pass invalid or malicious URLs.
No error handling guidance. Description does not explain what happens on network failure, timeout, invalid HTML, or blocked URLs. LLMs have no recovery path.
Security concerns unaddressed. No mention of rate limiting, credential leakage in responses, or sanitization of malicious content. Tool could expose sensitive data or be abused for DoS.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | <=2025-11-25 | v2 |
Description lacks context for tool selection. Does not explain when to call scrape_url vs. other research methods, what output format to expect, or dependencies (e.g., 'requires public URL').