Loading ToolBench
AWS Fault Injection Service (FIS) MCP Server - supports uvx execution
The server provides 10 tools with clear action verbs and basic descriptions. However, critical gaps in parameter descriptions, input schema documentation, output schema specification, and error handling significantly reduce quality. Most tools lack per-parameter descriptions, making it difficult for LLMs to understand parameter semantics. Output schemas are not formally documented. Error handling is minimal, most functions return error strings rather than structured error objects with recovery guidance. The server implements a write-mode gating mechanism (require_write_mode decorator) which is a positive security practice, but lacks permission-based access control and audit logging.
Create a new AWS FIS experiment template.
Delete an AWS FIS experiment template.
Generate an example AWS FIS experiment template with sample configuration.
Get detailed information about a specific AWS FIS experiment.
Get detailed information about a specific AWS FIS experiment template.
List all available AWS FIS action types.
CRITICAL: Output schemas not documented. Tools return JSON strings, but the structure of those strings (fields, types, nested objects) is not specified anywhere in the code or docstrings. LLMs cannot plan downstream tool calls or extract values without knowing the response structure.
CRITICAL: Most input parameters lack descriptions. The 'region' parameter appears in every tool but has only a trivial description 'AWS region to query (default: us-east-1)'. The 'targets' and 'actions' parameters in create_experiment_template are described as 'Dictionary of targets configuration' and 'Dictionary of actions configuration', vague and unhelpful. LLMs cannot infer the nested structure or valid keys without detailed parameter documentation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 64 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 57 | - | v1 |
List all AWS FIS experiment templates in the specified region.
List all AWS FIS experiments in the specified region.
Start a new AWS FIS experiment based on an experiment template.
Stop a running AWS FIS experiment.
HIGH: Error handling is minimal and unstructured. All error responses are plain strings (e.g., 'Error listing experiment templates: {str(e)}'). No error categorization (retryable vs. fatal), no recovery guidance, no structured error objects. LLMs cannot determine whether to retry, ask the user, or give up.
HIGH: No pagination support. list_experiment_templates and list_experiments do not accept limit or offset parameters. If a region has thousands of experiments, the tools will attempt to return all of them, exhausting the context window and causing token waste. The rubric baseline recommends capping results at 20-50 items with pagination controls.
MEDIUM: Missing tool annotations. The server does not use readOnlyHint, destructiveHint, or idempotentHint in tool definitions. Although risk classifications (READ_ONLY, WRITE, REVERSIBLE, DESTRUCTIVE) are provided in the server description, they are not reflected in the actual tool registration or accessible to clients via tool metadata.
MEDIUM: Vague parameter descriptions in create_experiment_template. The 'targets' parameter is described as 'Dictionary of targets configuration' with no hint about what keys are valid, what structure is expected (e.g., are the values objects, arrays, strings?), or how to know if the configuration is correct. Same issue for 'actions' and 'stop_conditions'. LLMs will guess or pass invalid structures.
MEDIUM: No audit logging or permission gates beyond write-mode gating. The server implements require_write_mode to disable all write operations by default, but there is no per-user permission checking, no audit trail logging which user called which tool, no scope declarations (e.g., 'read:fis', 'write:fis'), and no compliance traceability. In production, agent actions must be logged for debugging and regulatory compliance.
LOW: No dry-run or confirmation support for destructive operations. The delete_experiment_template tool will immediately delete a template without an intermediate confirmation step or dry-run preview. If an agent hallucinates the wrong template_id, the damage is irreversible. The rubric recommends confirmation workflows for irreversible operations.
LOW: No idempotency support for start_experiment. The tool accepts an optional client_token parameter but the code does not show how or if it is used to implement idempotent behavior. If an LLM retries start_experiment due to a transient error, duplicate experiments may be created.