Reference implementation for MCP and LLM gateway assessment tasks including a strict stdio MCP server, authenticated MCP HTTP gateway, streaming PII guardrail, and rate-limited model router
Server defines 4 tools with complete JSON schemas and descriptions. Two primary tools (get_customer_record, trigger_refund) have excellent input validation using Pydantic with strict type constraints, pattern matching, and field descriptions. However, two mock tools (public_echo, admin_reset_key) lack parameter descriptions and have minimal descriptions. Output schemas are not documented. Error handling is present but generic. Naming follows verb_noun convention well. Overall solid for a reference implementation but gaps in documentation and output schema definition prevent higher scores.
Mock privileged key rotation
Return a customer record for a validated customer ID.
Echo supplied text
Create a refund after strict argument validation.
Mock tools lack parameter descriptions. public_echo has 'text' parameter with no description; admin_reset_key has empty properties. LLMs cannot infer parameter meaning without explicit descriptions.
Output schemas not documented. Tools return structured data (customer records, refund confirmations) but LLMs have no schema definition for expected response fields. Agents cannot plan downstream calls without knowing what fields are returned.
Tool descriptions for mock tools are too brief (19-27 chars). 'Echo supplied text' and 'Mock privileged key rotation' lack context for when/why to use them. Baseline for A+ tools is 50-200 chars.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2025-06-18+ | v2 |
Error handling returns MCPError with validation details but does not guide recovery. When trigger_refund fails validation, the error includes constraint violations but no suggestion for next steps (e.g., 'Check amount is positive and reason is 10-500 chars').
public_echo tool is generic and does not follow single-responsibility principle. 'Echo supplied text' is a test/mock tool with no clear business purpose. In production, such tools should be removed or replaced with domain-specific operations.