Loading ToolBench
AWS Infrastructure as Code helper server with Terraform management, validation, drift detection, best practices checking, and architecture diagram generation
aws-iac-helper exhibits significant structural and documentation gaps across all dimensions. Tool names follow verb-noun conventions (generate_, validate_, detect_, check_) which is positive, but descriptions are present but minimal (50-100 chars), falling below the 194-char baseline for A+ tools. Input schemas ARE visible and properly typed (string, object), meeting baseline requirements, but output schemas are completely undocumented, all tools return plain strings rather than structured objects. Error handling is present but non-actionable (returns raw error strings without recovery guidance). No tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications. Parameter descriptions exist but are extremely brief (10-30 chars), offering minimal guidance to LLMs. The generate_terraform_template tool embeds example configuration keys directly in error messages ('Missing required configuration parameter'), which violates the pattern against example values in descriptions. Most critically, all tools return unstructured text blobs rather than machine-parseable JSON with typed fields, forcing LLMs to parse natural language output, an anti-pattern documented in mxe:response-shaper and mxe:strip-api-responses. This server would fail production code review due to lack of structured outputs and weak parameter documentation.
Check Terraform templates for best practices and security issues.
Detect infrastructure drift compared to the Terraform state.
Generate an architecture diagram from Terraform templates.
Generate a Terraform template for the specified AWS resource.
Validate a Terraform template file.
All tools return unstructured plain-text strings instead of documented JSON schemas with typed fields. LLMs cannot extract structured data or chain tool calls reliably. Example: detect_drift returns a formatted narrative string rather than {drift_detected: bool, resources_to_add: [string], resources_to_modify: [string], resources_to_destroy: [string]}
Parameter descriptions are trivially short (10-30 chars) and lack guidance on valid values, ranges, or dependencies. Example: 'template_path' described as 'Path to the Terraform template file or directory' (57 chars), does not specify accepted formats, whether it must exist, or what happens if both file and directory are provided.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 38 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
No tool annotations (readOnlyHint/destructiveHint/idempotentHint) despite explicit risk classifications in source (Risk: READ_ONLY, Risk: WRITE). detect_drift runs 'terraform plan' which may create state lock files; generate_architecture_diagram writes to disk, these side effects are not declared in MCP protocol terms.
Error handling returns raw error strings without actionable recovery guidance. Example: validate_terraform_template returns 'Failed to initialize Terraform: <stderr>', does not tell LLM whether error is retryable, what to try next, or whether it signals a user-fixable condition (e.g., missing terraform binary).
configuration parameter in generate_terraform_template is declared as generic 'object' type with no schema constraint. LLM has no visibility into required fields, types, or structure. EC2 template requires {ami, instance_type, name}; S3 requires {bucket_name, name}; RDS requires {storage, db_name, engine, engine_version, instance_class, username, password}, none of this is schema-encoded.
Tool descriptions are under the 194-char baseline for A+ tools. check_best_practices described in 60 chars ('Check Terraform templates for best practices and security issues.'), does not explain what 'best practices' means, whether tool is read-only or modifying, or what output format to expect.
No pagination or result-limiting mechanism documented. If terraform templates contain hundreds of resources, detect_drift and check_best_practices could return massive unstructured lists, exhausting context window. No limit, offset, or next_cursor parameters visible.
No idempotency guarantees documented. detect_drift creates a temporary tfplan file but does not clean it up or guarantee same result on retry. generate_architecture_diagram writes to output_path, repeated calls will overwrite. Agents retry on ambiguous failures; non-idempotent tools risk duplicates or side effects.
Subprocess calls execute 'terraform' and 'terraform init' without path validation or timeout. If terraform is not installed or takes >30s, tool hangs or fails with unclear error. No guidance for user on prerequisites (terraform must be installed, working_dir must exist, AWS credentials must be configured).