Loading ToolBench
MCP-backed Terraform assistant with ephemeral-values compliance.
This server has moderate structural issues across all tools. Tool naming follows verb-noun convention (scan_, get_, generate_), which is positive. However, parameter descriptions are missing entirely from the input schema definitions visible in the code. The schemas are present but lack descriptive text for parameters, only 'path', 'state_path', 'use_schema', 'provider', 'resource', and 'name' appear with minimal context. Tool descriptions themselves are present but brief (10-60 chars), below the 194-char baseline. No output schema is documented. Error handling in the handlers (ValueError for missing required params, try-catch for RegistryError) is minimal, handlers do not return actionable recovery guidance. The server lacks pagination, idempotence hints, and structured output documentation.
Generate a Terraform snippet for a supported resource.
Fetch provider metadata from Terraform Registry.
Run compliance checks over Terraform configs and optional state.
Missing parameter descriptions in JSON schemas. Input schemas define 'path', 'state_path', 'use_schema', 'provider', 'resource', 'name' with types only; no description text explaining what each parameter controls or what format is expected. Pattern requires every parameter to have a description for LLM understanding.
No output schema documented. Tools return structured objects (dict/JSON) but callers have no formal schema describing the fields, types, or required keys in responses. LLMs cannot plan downstream calls or validate extracted data without documented return types.
Tool descriptions are too brief (10-60 chars). Baseline is 194 chars. Descriptions lack context on WHEN to use the tool, what it returns, and how it differs from similar tools. E.g., 'Run compliance checks over Terraform configs and optional state' does not explain what compliance rules are checked, what the output contains, or when to call it vs. generate_snippet.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 45 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 36 | - | v1 |
Error handling is minimal and non-actionable. Handlers raise ValueError and catch RegistryError with a generic string response, but do not guide LLM recovery. E.g., 'path is required' tells the LLM to add a path param, but 'Invalid provider' does not tell it which providers are valid or how to discover them. No recovery guidance per pattern:recovery-guide.
No indication of idempotence, read-only status, or destructive intent in tool metadata. Tools are READ_ONLY (scan, get, generate), but schemas lack tool annotations (readOnlyHint, destructiveHint, idempotentHint) to signal intent to agents. LLMs treat all tools as potentially dangerous without explicit hints.
No pagination or result limits documented. scan_terraform may return large compliance reports; get_provider_metadata fetches Registry metadata; generate_snippet produces code. No mention of pagination params, max result sizes, or truncation behavior. Large unbound responses waste context and degrade LLM reasoning.
Parameter 'path' in scan_terraform lacks format guidance. Should specify: is it absolute or relative? Does it accept both files and directories? What file extensions are valid? Current description 'Path to a Terraform file or directory' is vague; LLMs may pass invalid paths or guess the expected format.
Parameter 'provider' in get_provider_metadata has enum constraint but no description text. LLMs can read the enum list, but schema does not explain what the tool returns or why only these 12 providers are supported. Is azure different from azurerm? The description should clarify.