Loading ToolBench
Open-source observatory of the agentic web: weekly AI-readiness audits of well-known sites, published as a static index. Includes field-test episode production with autonomous task generation and agent evaluation.
Six tools with mixed quality. All have descriptions (10-200+ chars), but schemas are incomplete or missing. visitPage and searchWeb have detailed implementations with proper error handling and structured PageView/SearchHit returns. However, tool definitions lack explicit JSON Schema registration in MCP format, schemas are inferred from code rather than formally declared. Parameters lack type annotations in schema form. propose_tasks has a complex nested schema visible in code but not formally registered. No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Error handling is present in implementation but not surfaced in tool definitions. Naming is clear (verb_noun pattern), but parameter descriptions are minimal.
Evaluate a domain's AI-readiness across 6 checks: llms.txt presence, AI crawler policy, agent-parseable content, structured data, sitemap discoverability, and everyday agent task support. Returns comprehensive SiteEvaluation with score, grade, posture, and detailed check results.
Propose the tasks for this week's episode. Call exactly once, after your research.
Web search (DuckDuckGo). Returns titles, URLs and snippets. Use 3-6 focused queries about this week's news.
Web search using DuckDuckGo's no-JavaScript results page. Returns titles, URLs and snippets. Unwraps redirect links to expose true URLs. Detects bot challenges and returns structured SearchHit results.
Fetch a web page (plain GET, no JavaScript) and read its text. Use it to confirm a story or a real URL before building a task on it.
Fetch a web page (plain GET, no JavaScript) and read its text. Returns page title, stripped text content, and extracted links. Pages are capped at 5000 characters of text and 40 links. Detects bot walls and returns structured PageView result.
Tool definitions lack formal JSON Schema registration. Schemas are inferred from code (visitPage returns PageView, searchWeb returns SearchHit) but not explicitly declared in MCP tool registration. This prevents LLMs from seeing structured output types before calling tools.
Parameter descriptions are minimal or missing. 'rawUrl' in visitPage has description 'URL to visit (http or https)' but lacks format constraints (must be http/https, max length, etc.). 'domain' in evaluateSite lacks any description. LLMs cannot infer valid input ranges.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All six tools are read-only (GET requests, no state mutation), but this is not declared in tool metadata. Agents cannot infer safety properties without explicit annotations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
Duplicate/overlapping tools. 'visitPage' and 'visit' both fetch web pages with nearly identical functionality. 'searchWeb' and 'search' both perform DuckDuckGo searches. LLMs will waste reasoning cycles deciding between them. Consolidate to one canonical tool per operation.
Error handling not surfaced in tool definitions. Code implements timeout detection, bot wall detection, and structured error responses (PageView.error, SearchHit error field), but tool descriptions do not document error cases or recovery guidance. LLMs cannot plan error recovery.