Loading ToolBench
Evaluates geodistances, address discrepancies, and checkout ticket size risks for fraud prevention.
This MCP server has a single tool 'audit_transaction' with a clear verb-noun name and basic schema. However, it suffers from significant definition quality gaps: the tool description (82 chars) is functional but lacks depth on WHEN to use it and what downstream actions follow. The schema is present with typed parameters, but lacks constraints (enums, ranges, patterns) that would guide LLM input validation. Parameter descriptions are minimal (16-66 chars) and lack format specifications or validation guidance. Most critically, there is NO output schema documentation visible in the MCP registration, only the Python implementation shows the structure. The tool lacks error handling guidance, recovery paths, and confirmation mechanisms for a sensitive fraud-detection operation. The implementation is straightforward (no external dependencies, pure Python) but the MCP interface itself lacks production-grade polish.
Evaluates fraud risk by analyzing shipping country, billing country, IP country mismatch, and order amount.
Output schema is NOT documented in MCP tool definition. Only the Python implementation reveals the response structure (fraud_score, status, reasons, requires_manual_verification). The MCP server must declare the output schema so LLMs know what fields to expect and can plan downstream actions.
Parameter descriptions lack validation constraints. E.g. 'order_amount_usd' should state 'positive number in USD, typically 0-999999' and country codes should specify 'ISO 3166-1 alpha-2 code (e.g. US, CA, GB)' to guide LLM input and enable validation.
No error handling or recovery guidance in tool definition. A fraud-detection tool flagging or declining transactions is sensitive, the description should guide agents on next steps: 'Returns status="flagged" for scores 30-59, agents should request manual verification. Returns status="declined" for scores 60+.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Tool description is too generic (82 chars) and lacks WHEN-to-use context. It states WHAT but not when a fraud check should be triggered or how the output (declined/flagged/approved) drives downstream actions like manual review or payment block.
No idempotency or confirmation pattern. This is a read-only audit tool so replay is safe, but this should be explicitly documented (readOnlyHint annotation). For fraud decisions, agents should know which calls are safe to retry.