Loading ToolBench
Five tools with comprehensive input schemas and descriptions. Strong naming conventions (all start with action verbs: search, get). Tool annotations present (readOnlyHint, idempotentHint). Output schemas documented for all tools. However, descriptions are sometimes generic (e.g., scp_search 'via SCP Data API'), lack dependency hints, and error handling guidance is missing. Parameters are well-typed with Zod, but some descriptions could be more actionable for LLM selection. Security logging is present (AuditLogger, rate limiting), and responses include proper attribution/licensing metadata. Overall solid foundation with room for LLM optimization.
Generate CC BY-SA 3.0 attribution text for a page.
Get page content in markdown/text/html/wikitext.
Get a page by link, SCP number, or Wikidot page_id.
Get related pages based on references/hubs.
Search SCP Wiki pages via SCP Data API (CC BY-SA 3.0).
scp_search and scp_get_page both accept 'link' as optional parameter; scp_get_page also accepts 'scp_number' and 'page_id'. Parameter descriptions ('Page slug', 'SCP number', 'Wikidot page id') exist but lack clarity on mutual exclusivity or precedence. LLM may pass multiple and receive ambiguous behavior.
Tool descriptions lack WHEN/WHY guidance. E.g., 'Search SCP Wiki pages via SCP Data API (CC BY-SA 3.0)' does not explain when to call scp_search vs scp_get_page vs scp_get_related. LLMs cannot distinguish intent without contrastive context.
No error recovery guidance in tool descriptions. If scp_search returns zero results or scp_get_page returns 404, tool does not suggest next steps (e.g., 'try a broader search query' or 'call scp_search to find the page first'). Agents lack context to self-correct.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2025-06-18+ | v2 |
scp_get_content accepts 'format' as required parameter but tool description does not explain what each format is best for (when to choose markdown vs text vs html vs wikitext). Description states 'Output format' without guidance.
scp_get_related tool description ('Get related pages based on references/hubs') is vague. Does not clarify how 'related' is determined (backlinks? forward references? same series? same tags?). LLM cannot predict output quality.
scp_search 'limit' parameter description says 'default 20, max 50' but schema does not enforce max constraint (.number().int().optional() lacks .max(50)). If LLM passes 100, backend may reject or silently truncate without clear error.
Response includes 'content_is_untrusted: true' flag for scp_search and other tools (indicating user-generated content), but tool descriptions do not warn LLM about this or suggest sanitization/validation steps. Risk of prompt injection via untrusted wiki content.