AI customer support agent for Meridian Electronics that browses products, checks stock, looks up customer order history, and places orders using MCP tools from a remote battle_tank server
Critical gaps across all dimensions. Tool definitions exist only as text in a system prompt (src/prompts/index.py), not as explicit MCP tool registrations with schemas. No input schemas visible for 7 of 8 tools. Only create_order has a partial schema shown. Descriptions are present but generic (10-80 chars, below the 194-char baseline). No parameter descriptions for most tools. No output schemas documented. No error handling guidance. This is a LangGraph application consuming tools from an external MCP server (battle_tank), not an MCP server itself, tool definitions are inferred from prompt text, not formally registered.
place a new order (requires customer_id, SKU, quantity, unit_price, currency)
look up a customer's profile and shipping address by customer ID
get full details and line items for a specific order
get price, stock, and details for a specific SKU
view a customer's order history, filter by status if needed
browse inventory, optionally filter by category or active status
find products by keyword when the customer describes what they want
Tool definitions are text-only in system prompt, not formally registered MCP tools. No explicit tool registration with JSON Schema visible in codebase. Tools are inferred from prompt descriptions, not declared via MCP protocol.
No input schemas visible for 7 of 8 tools. Only create_order shows a partial schema with 5 parameters. Remaining tools lack documented parameter types, descriptions, and constraints. LLMs cannot infer valid inputs.
Descriptions are generic and under 50 chars for most tools (baseline 194 chars). 'browse inventory, optionally filter by category or active status' lacks WHEN to use, WHAT it returns, and dependencies. No parameter descriptions at all.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 29 | 2026-07-28+ | v2 |
authenticate a customer with their email and 4-digit PIN before placing an order
No output schemas documented. LLMs cannot plan downstream tool calls or extract required fields (e.g., does get_product return stock_level or inventory_count?). No pagination guidance for list_* tools.
No error handling guidance. System prompt says 'Never guess product details' but tools lack recovery hints. If verify_customer_pin fails, LLM has no guidance on retry strategy or user-fixable vs fatal errors.