Loading ToolBench
Comprehensive onboarding pipeline for Unified Commerce Platform (UCP) integration. Runs audit, profile generation, catalog mapping, validation, and optional ACP feed export and sandbox checkout server generation for merchants.
This server exposes 6 tools via STDIO with basic schemas and descriptions, but falls short of production quality. All tools have descriptions (10-200 chars, baseline 194), and input schemas are present with typed parameters. However, descriptions lack LLM-optimization guidance (e.g., 'when to use', 'what happens'), parameter descriptions are minimal or missing context, output schemas are undocumented, and error handling is absent. No tool annotations (readOnlyHint/destructiveHint), no pagination support, no recovery guidance. Tools like generate_profile and generate_api perform state-changing operations but lack confirmation patterns or dry-run support. The naming is verb-forward (audit_site, generate_profile, map_catalog) but some names are generic (validate_ucp could be validate_ucp_profile). Per-tool average: 42/100.
UCP Readiness Audit — scans a merchant website and scores UCP readiness. Checks for UCP profile, platform detection, structured data, payment providers, and API accessibility.
Export a UCP-style catalog JSON file to an OpenAI ACP product feed payload. Normalizes product data, variants, pricing, media, and categories for OpenAI ACP compatibility.
Generate a sandbox FastAPI UCP checkout server from profile and catalog JSON. Creates a complete FastAPI application with UCP endpoints for catalog search, lookup, and checkout operations.
Generate a UCP business profile (/.well-known/ucp) for a merchant. Creates profile with services, capabilities, and payment handlers based on domain, business name, payment provider, and transport type.
Map merchant product data to UCP catalog schema format. Supports Shopify (products.json), CSV, and raw JSON sources. Performs field/structure validation and generates mapping report.
Validate UCP integration. Checks merchant website for UCP profile compliance and generates validation report.
Output schemas undocumented. Tools return files (markdown reports, JSON) but LLM cannot infer structure, field names, or downstream chaining. E.g., audit_site returns audit.json but schema is invisible.
Parameter descriptions are minimal or absent. E.g., generate_profile 'caps' param says 'Comma-separated capabilities' but does not list valid values (checkout, cart, catalog, fulfillment, discount, order). LLM must guess or hallucinate.
No error handling or recovery guidance. Tools fail silently or return generic errors. E.g., if audit_site cannot reach a URL, LLM has no guidance on retry, fallback, or next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 38 | 2026-07-28+ | v2 |
State-changing tools (generate_profile, generate_api, export_acp_feed) lack confirmation or dry-run patterns. Agents cannot preview results before committing. No idempotency guarantees.
Tool descriptions lack 'when to use' and 'what happens' context. E.g., validate_ucp says 'Validate UCP integration' but does not explain when to call it vs audit_site, or what compliance checks it performs.