Loading ToolBench
Generate AWS CLI commands from plain English with built-in safety checks
Three tools with basic descriptions but critical gaps in schema documentation and parameter clarity. generate_aws_cli has a lengthy description (250+ chars) but no visible input schema in the code, only a bare 'query: str' parameter. health_check and list_supported_services_mcp lack parameter descriptions entirely. No output schemas documented. No error handling guidance. No enum constraints or validation rules. Naming is reasonable (verb_noun pattern), but parameter documentation is absent, violating the 100% baseline for A+ tools.
Generate AWS CLI command from natural language query. This tool NEVER executes commands. It only generates command strings for human review and manual execution. Returns a stable JSON payload (see docs/MCP_CONTRACT.md): - command: the AWS CLI string to copy and run - intent: detected operation - service: AWS service name - safety.level: SAFE | MUTATING | SECURITY_SENSITIVE | DESTRUCTIVE - safety.requires_confirmation: bool - execution.allowed: always False - execution.mode: always "manual"
Health check endpoint for the MCP server
List all supported AWS services
No input schemas visible in source code. generate_aws_cli declares 'query: str' but no JSON Schema with type/description. health_check and list_supported_services_mcp have empty input objects with no documentation.
No output schemas documented. generate_aws_cli returns a complex JSON payload (command, intent, service, safety.*, execution.*) but the response structure is not formally specified. LLMs cannot plan downstream tool calls without knowing return fields.
Parameter descriptions missing. 'query' parameter in generate_aws_cli has no description explaining format, length limits, or examples of valid queries. health_check and list_supported_services_mcp have no parameters documented at all.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 39 | 2026-07-28+ | v2 |
No error handling guidance. If generate_aws_cli receives an unsupported service or malformed query, the response does not indicate what the LLM should do next (retry, ask user, try alternative tool). No recovery paths documented.
health_check and list_supported_services_mcp are utility tools with minimal descriptions (20-30 chars). Descriptions do not explain when to call them or what they return. health_check description is 'Health check endpoint for the MCP server', too generic.