Loading ToolBench
Control plane MCP that dedups scanner output, rate-limits hosts, and runs non-destructive proofs.
Server has 13 tools with complete input schemas and descriptions. Naming follows verb_noun convention (begin_engagement, draft_waves, execute_wave, etc.). Descriptions are present and substantive (avg ~120 chars), explaining WHAT each tool does and WHEN to use it. However, output schemas are not documented, responses are wrapped in {ok, data, error} but the structure of 'data' is never specified. Parameter descriptions are adequate but lack constraint details (ranges, enums, formats). Error handling returns structured {ok, error, message} but lacks recovery guidance. No tool annotations (readOnlyHint, destructiveHint) despite clear risk classifications. Security-sensitive parameters (tester_account_a, tester_account_b, expected_marker) are exposed as tool parameters rather than server-side injected.
Explain candidate impact, missing evidence, controls, and safe proof options.
Register exact/wildcard scope and bounded traffic policy. Default mode is self-attested: calling begin_engagement is the authorization. Set KEEL_APPROVAL_FILE for optional strict/team manifests. The legacy operator_confirmed argument is ignored and cannot grant approval.
Request cancellation of a queued or running scanner process.
Describe an allowlisted proof without sending traffic.
Draft reachability plus one micro-wave per reviewed template; no traffic.
Return recent append-only application audit events for one engagement.
Output schemas not documented. Tools return {ok, data, error} but the structure of 'data' payload is never specified. LLMs cannot plan downstream calls or extract fields without knowing response structure.
Security-sensitive parameters exposed as tool inputs. tester_account_a, tester_account_b, expected_marker, proof_target_ref, and session_a/session_b should be server-side injected via environment or vault, not passed as parameters. Agent traces log all parameters, credentials leak into logs.
No tool annotations despite clear risk classifications. Tools marked WRITE (begin_engagement, execute_wave, state_impact, execute_proof) and REVERSIBLE (cancel_wave) lack destructiveHint/idempotentHint annotations. LLMs cannot distinguish safe reads from destructive writes without explicit hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Report registered engagements, cooldowns, and pending waves.
Run a proof bound to an operator-approved target, credentials, and canary hash.
Start one admitted wave in the background and return its persistent job.
Return hunter-relevant cards. Informational and hardening are hidden by default.
Start a background micro-wave using only the originating template.
Record hunter impact_class and preconditions on a card.
Return one job, or list persistent jobs/waves when job_id is omitted.
Parameter constraints not documented. Numeric parameters (requests_per_second, max_parallel_hosts, max_wave_seconds, max_wave_requests, max_engagement_requests, max_response_bytes, max_proof_requests) lack min/max bounds in descriptions. LLMs can pass absurd values (negative, zero, unbounded).
Error responses lack recovery guidance. Tools return {ok: false, error: 'KeelError', message: '...'} but do not tell LLMs what to do next. A 'card not found' error should suggest 'Call query_cards() to list available cards' or similar.