AWS Fault Injection Service (FIS) FastMCP Server for managing FIS experiments and templates
Server has 10 tools with basic descriptions and input schemas, but significant quality gaps limit production readiness. All tools have descriptions (10-50 chars, below the 50-200 char baseline for LLM optimization), and input schemas are present but minimal. Parameter descriptions are generic ('AWS region to query') and lack actionable constraints. Error handling returns raw exception strings instead of recovery guidance. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels. Output schemas are not documented, LLMs cannot predict response structure. Tools like create_experiment_template accept complex nested objects (targets, actions, stop_conditions) with no schema definition or validation guidance.
Create a new AWS FIS experiment template.
Delete an AWS FIS experiment template.
Generate an example AWS FIS experiment template.
Get detailed information about a specific AWS FIS experiment.
Get detailed information about a specific AWS FIS experiment template.
List available AWS FIS action types.
Descriptions are too short (10-50 chars vs. 50-200 char baseline). 'List all AWS FIS experiment templates in the specified region' lacks context on WHEN to call this vs. get_experiment_template, and does not explain what structure is returned.
create_experiment_template accepts complex nested objects (targets, actions, stop_conditions) as type 'object' with no schema definition, validation rules, or examples. LLMs cannot construct valid payloads without explicit constraints.
Error handling returns raw exception strings ('Error listing experiment templates: {str(e)}') instead of actionable recovery guidance. LLMs cannot determine if errors are retryable, user-fixable, or fatal.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
List all AWS FIS experiment templates in the specified region.
List all AWS FIS experiments in the specified region.
Start a new AWS FIS experiment based on an experiment template.
Stop a running AWS FIS experiment.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. delete_experiment_template is DESTRUCTIVE but unmarked; start_experiment is WRITE but unmarked. Agents cannot reason about side effects.
Output schemas are not documented. Tools return JSON strings but LLMs cannot predict field names, types, or structure. E.g., get_experiment returns 'state' as an object, what fields does it contain?