Verifies and replays signed payment webhooks (Stripe/Adyen/PayPal/Plaid) locally, catching signature, idempotency, and replay-attack bugs.
Single tool 'webhookvty_scan' has a reasonable description (127 chars, within 10-1024 range) but critical schema deficiencies. Input schema is present but minimal: only 'target' parameter with basic string type and description. No output schema documented anywhere, LLMs cannot infer what fields to expect from the JSON response. Parameter lacks constraints (min/max length, format, path validation rules). Tool name 'webhookvty_scan' is verb-noun but generic; 'scan' alone doesn't convey the specific domain (webhook verification). No error handling guidance, no pagination support, no per-item success/failure reporting. The description mentions 'Stripe/Adyen/PayPal/Plaid' but doesn't explain when to use this tool vs. other security scanners, or what 'findings' structure contains. Baseline: 100% of A+ tools document return types; this tool does not.
Verifies and replays signed payment webhooks (Stripe/Adyen/PayPal/Plaid) locally, catching signature, idempotency, and replay-attack bugs.. Returns JSON findings.
No output schema documented. LLMs cannot plan downstream actions or extract fields from the JSON response. Baseline: 100% of A+ tools have documented return types.
Input parameter 'target' lacks constraints. No min/max length, no format specification, no validation rules for path traversal or symlink handling. Description does not explain expected format (absolute path? relative? glob pattern?).
Tool description does not explain WHEN to use this tool or what makes it distinct from generic code scanners. Mentions payment providers but doesn't clarify: does it only work on webhook handler code? Does it require specific file patterns? What does 'replay-attack bugs' mean to an LLM unfamiliar with webhook security?
No error handling guidance. If 'target' is not a valid path, or contains no webhook code, or scan fails, what does the tool return? How should the LLM recover? Baseline: error responses must tell the LLM what to do next.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Tool name 'webhookvty_scan' is generic. 'scan' could mean security scan, code scan, or health check. Better: 'verify_webhook_signatures' or 'audit_payment_webhooks' to clarify domain and action.