Loading ToolBench
Static MCP host: run untrusted MCP tool.js files sandboxed in QuickJS-wasm (Cloudflare Workers or Node), with explicit host capabilities (fetchOrigin, callInternal, extraCapabilities) and llms.txt executable-skills parsing.
mcpwasm exposes 19 tools with explicit schemas and descriptions. Most tools have clear verb-noun naming (get_book, create_order, search_catalog) and reasonable descriptions (avg ~150 chars). However, several critical gaps reduce quality: (1) duplicate tool names (buy_creator_access, check_license, create_order appear twice with different implementations), creating ambiguity for LLM selection; (2) parameter descriptions are sparse, many lack guidance on format, range, or constraints (e.g., 'qty' says 'integer >= 1' in description but no min/max in schema); (3) output schemas are undocumented, tools return structured objects but no formal schema is declared; (4) error handling is minimal, most tools return {ok:false, status, error} but lack recovery guidance; (5) test fixtures (busy_loop, corrupt_skill) are included in the tool list, polluting the production interface. Naming is generally strong (90% start with action verbs), but composition and error patterns are weak.
TEST FIXTURE for gateway interrupt: handler runs an infinite while loop. A conforming gateway with a QuickJS sandbox using interrupt/timeout MUST abort this. Do NOT call outside a sandbox.
Start the purchase of a creator license to create items on this platform. Returns the paylink for the HUMAN to pay. After payment the paylink page shows the license token: the human gives it to you for create_item. Free to call.
Start the purchase of a creator license to list products ($19 for 25 listings, 30 days). Returns the paylink for the HUMAN to pay. After payment the paylink page shows the license token: the human gives it to you for create_product. Free to call.
Check a creator license token: status, listings left, expiry. Free. Useful to verify a token before trying create_product, or to remind the human how many listings they have left.
Check a creator license token: status, creations left, expiry. Free. Useful before create_item, or to tell the human how many creations they have left.
TEST FIXTURE for gateway robustness: this tool.js is valid and served correctly, but the tool_sha256 declared for it in /llms.txt is intentionally WRONG. A conforming gateway MUST exclude this skill from discovery.
Duplicate tool names across namespaces: buy_creator_access, check_license, and create_order appear 2-3 times with different implementations (bookstore vs byoa-kit vs marketplace). LLMs cannot disambiguate which variant to call, causing incorrect tool selection and failures.
Output schemas are undocumented. Tools return structured objects (e.g., {ok:true, order_id, book_id, qty, remaining_stock}) but no formal JSON Schema is declared. LLMs cannot plan downstream calls or extract fields reliably without knowing the response structure.
Parameter constraints are described in text but not enforced in schema. E.g., qty='integer >= 1' is stated in description but schema has no minimum/maximum. LLMs may pass invalid values; validation happens at runtime with generic error messages.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
PAID TOOL (creator license): create a new item on the platform (name, price, optional description/stock). Buy access with buy_creator_access, then pass the license token as access_token. Without a valid token returns needs_payment. REAL-WORLD EFFECT: always confirm name and price with the human first.
Create an order for a product (decrements stock atomically). Returns {ok:true, order_id, sku, qty, total, remaining_stock, order_status}. IDEMPOTENT: pass a client_order_id (any string unique to this purchase intent, e.g. a UUID) and retries return the SAME order instead of duplicating it. Returns {ok:false, status:409} when the SKU is unknown or stock is insufficient.
Create an order for a book (decrements stock atomically). Returns {ok:true, order_id, book_id, qty, remaining_stock} on success, or {ok:false, status:409, error, ...} when the book does not exist or stock is insufficient.
Get full details of a single book by its numeric id. Returns {found:false} when the book does not exist.
Fetch one of the 4 published llms-txt-skills documents by name. Returns {name, length, content} with content truncated to 4000 chars.
Get full details of one item by numeric id: name, description, price, stock. Returns {found:false} when the id does not exist.
List the 4 published llms-txt-skills documents with title and path. Static, no network fetch.
List items from the platform catalog (filter by free text, limit 10). Public read: no approval needed.
Search the bookstore catalog by text (matches title or author), optionally filtered by genre and a maximum price. Returns up to 10 matching books.
BM25 search over the llms-txt-skills spec snapshot (4 documents). Returns hits as {text, score, title, concept_id}.
Return the current server time.
Return an inventory stock report: total number of titles, sum of stock across all titles, number of titles out of stock, and the top 3 titles by stock.
Sum two numbers a and b.
Test fixtures (busy_loop, corrupt_skill) are exposed as production tools. These are intentionally broken for gateway testing and should not appear in the tool list. They pollute discovery and confuse LLMs about what is safe to call.
Error handling lacks recovery guidance. Tools return {ok:false, status:409, error} but do not tell the LLM what to do next (e.g., 'Try search_catalog() first' or 'Check stock_report() for availability'). Agents cannot self-correct.