Forward Deployed Engineer assessment: MCP server, MCP security gateway, streaming PII guardrail, model router.
Two well-defined tools with strong schema validation and clear descriptions. Both tools have complete JSON Schema with type constraints, patterns, and ranges. Descriptions are concise (40-60 chars) and action-oriented. Input validation is strict (customer_id pattern, amount bounds, reason minLength). Error handling distinguishes protocol errors (validation) from domain errors (business logic). However, output schema is not formally documented, responses are returned as structuredContent but the shape is inferred from code rather than declared. Tool composition is minimal (only 2 tools), limiting assessment of chaining patterns. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels.
Look up a single customer record by customer id.
Issue a refund against a customer's refundable balance.
Output schema not formally documented. Responses are returned as structuredContent (dict with refund_id, customer_id, amount, reason, status, remaining_refundable_usd) but the shape is inferred from code, not declared in tool definition. LLMs cannot plan downstream calls without knowing what fields to expect.
Tool annotations missing. trigger_refund is marked Risk: WRITE but has no destructiveHint in schema. get_customer_record is READ_ONLY but has no readOnlyHint. These hints help LLMs reason about safety and retry logic.
trigger_refund description lacks state-change clarity. Should explicitly state 'Issues a refund and deducts from the customer's refundable balance' to signal irreversibility. Current description 'Issue a refund against a customer's refundable balance' is passive and does not clearly indicate side effects.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 71 | 2025-06-18+ | v2 |
Error messages are domain-specific but lack recovery guidance. When refund is rejected (INSUFFICIENT_BALANCE, BELOW_MINIMUM), the error tells the LLM what went wrong but not what to do next. Should suggest 'Check refundable_balance via get_customer_record' or 'Reduce amount and retry'.
reason parameter description uses regex pattern but does not explain it in plain language. Description says 'At least 10 non-whitespace characters' but the pattern '^(?=(?:\s*\S){10})' is not self-evident.