Loading ToolBench
MCP Gateway server providing tools for Assistant, Broadcast, and Cart services with authentication and authorization based on API key scopes and tool surface classification
Mixed quality across tools. Positive: all 9 tools have descriptions and input schemas are present with parameter types. Negative: descriptions are generic and often repeat the same action pattern, parameter descriptions are minimal or generic, no output schema documentation, no error recovery guidance, and several parameters lack meaningful constraints. Tools 1-3 (manage_brand, manage_catalog, manage_category) use an identical action-switching pattern with overloaded parameters (action string + conditional json/code params), which violates single-responsibility and forces LLMs to reason about conditional logic. Tools 4-8 (data exchange) are repetitive, all using identical action/profile/format patterns. The get_country_details tool is the cleanest but still lacks output schema. Overall, definitions would not pass principal engineer review due to lack of actionable error messages, missing output schemas, and generic parameter descriptions.
Send a storefront assistant message and return the AI response, product cards, suggestions, intent, cache flag, and session message count. Optional user ID is sent only to AssistantService and is never echoed in the MCP response.
Send a storefront assistant message through the streaming chat RPC and return one aggregated JSON response with combined text deltas, product cards, suggestions, intent, session message count, and products found.
Clear the AssistantService conversation context for a storefront session.
Get conversation-aware assistant suggestions for a storefront session.
Cancel a pending broadcast.
POS-side operation: atomically transfer cart ownership to a cashier session using a claim code. The cashier types or scans the code shown by the customer. Returns NotFound if the code is unknown, FailedPrecondition if expired or already claimed. Requires storeId and cashierUserId (GUIDs).
Action-based tools force LLM conditional reasoning. manage_brand, manage_catalog, manage_category all use string action parameter with switch logic (create/update/delete/get/list), requiring the LLM to construct proper JSON and select the right parameter subset. This violates single-responsibility and creates opportunities for misuse. Pattern guidance: split into discrete tools (create_brand, update_brand, delete_brand, get_brand, list_brands) so each tool name clearly declares intent.
No output schemas documented. All 9 tools have no stated return type or field structure. LLMs cannot predict what fields will be available for downstream calls or data extraction. Example: manage_brand returns formatted data but schema is unknown, agent cannot reliably extract brand_id for use in other tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 0 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 33 | - | v1 |
Create and schedule a broadcast. targetConfig is audience/channel configuration, not contact PII.
Delete a broadcast.
Get a broadcast by ID. No customer contact PII is returned.
Get a single cart by its ID. Optionally scope to a storefront via storefrontId. Returns cart details with items and totals. Customer PII (phone, address) is omitted.
Issue an HMAC-signed claim code for a cart to enable phone-to-POS handoff. The customer shows the QR code or 4-letter code at the till; the cashier scans or types it to claim the cart. Idempotent: re-issuing for the same cart invalidates the previous code. Default TTL is 15 minutes; supply expiresAt to override.
List broadcasts with optional storefront, target type and status filters. No customer contact PII is returned.
List shopping carts with optional filtering. Provide storefrontId to scope to a storefront; omit to list across all storefronts. Filter by hasItems (true/false), softDeleteFilter (active/deleted/all). Supports pagination via page and pageSize.
Retry a failed broadcast.
Update broadcast content or schedule fields.
Generic parameter descriptions. Many parameters have descriptions like 'Brand JSON (for create/update)', 'Search text (for list)', 'Language code to filter translations', these are minimal and do not explain format, constraints, or valid values. LLMs cannot infer that json must be valid JSON string, not an object; that language should be an ISO code; or what fields are required in the JSON object.
No error recovery guidance. Validation code (e.g., 'code is required for get') returns generic error messages without recovery hints. Example: 'json is required for create/update' does not guide the LLM on what the JSON structure should be or where to find examples. Pattern guidance: errors should hint at recovery (e.g., 'json is required for create/update. Expected: {"code": "...", "translations": [{"languageCode": "...", "name": "..."}]}')
Repetitive tool definitions. customer_data, order_data, product_catalog_data, reference_data, review_data all follow identical structure (action/profile/format/content/language/search/page/pageSize), making the tool set feel less discoverable and harder for LLMs to differentiate. While not strictly a pattern violation, the repetition suggests these could be unified or better composed.
Missing constraints on enumerated parameters. 'action' parameter in manage_brand accepts values create|update|delete|get|list, but this is documented only in the description text, not as an enum in the schema. Schema lacks 'enum' field to restrict LLM choices. Same for 'profile' in customer_data (should be enum: [customers]), 'format' (should be enum: [json, csv, excel, xml]), and 'status' filters.
No pagination limit enforcement. Data exchange tools (customer_data, order_data, product_catalog_data, reference_data, review_data) accept unbounded pageSize parameter. LLM could request pageSize=10000, causing memory exhaustion or timeout. Pattern guidance: cap pageSize (e.g., 1 - 500) and state the limit in the description.
Unclear parameter requirements and relationships. In manage_brand, parameters json, code, search are conditionally required based on action value, but the schema marks them all as optional (required: false or omitted). An LLM might call delete action without a code, hitting a validation error instead of being prevented by schema. Pattern guidance: either split tools or document mutual exclusivity clearly with examples.
No documentation of JSON schema for nested structures. manage_catalog expects 'json' with nested categories [{...}], media [{...}], but no description of what fields categories or media require. LLM must guess structure, leading to malformed requests.
Naming inconsistency and ambiguity. manage_brand, manage_catalog, manage_category use the same pattern and could be confused for each other. Tool names do not start with a clear action verb (create_brand, update_brand would be clearer). get_country_details is the only verb_noun pattern that is clear.