Loading ToolBench
A payment processing gateway that aggregates multiple payment providers (Midtrans, Xendit) and handles checkout creation, webhook processing, and transaction reconciliation
PayLink is a custom HTTP payment processor implementing 5 tools with significant definition quality gaps. Schemas ARE present and properly typed (json decoder on models.CheckoutRequest, etc.), but descriptions are inconsistent, some tools have detailed descriptions while others are minimal. Parameter descriptions exist but lack constraint documentation (e.g., no enum for provider_preference, no bounds on amount despite validation code showing 999999999 max). Tool naming is acceptable (Checkout, HandleWebhook, GetTransaction) but lacks consistency, some are noun-based rather than verb-noun convention. No evidence of output schema documentation for downstream tool chaining. Error handling exists in code (validateCheckoutRequest) but does not follow the recovery-guide pattern; errors are generic HTTP responses without actionable next steps. The tool definitions appear to be inferred from handler function signatures rather than explicit registration, which under the scoring rules caps per-tool scores at 50.
Creates a new payment checkout by validating the checkout request, instantiating the appropriate payment provider adapter, and returning a checkout URL and provider transaction ID
Retrieves transaction status by transaction ID; currently returns pending status but placeholder for database query
Processes incoming webhook notifications from payment providers by verifying the signature, extracting the provider from the URL path, and enqueuing the webhook job for asynchronous processing
Returns server health status endpoint
Returns Prometheus-formatted metrics for the PayLink server including uptime, request counts, checkout counts by provider, webhook counts, and average latency
Tool definitions inferred from handler functions, not explicitly registered. Source code shows http.HandleFunc calls mapping routes to handlers, but no explicit tool registration with MCP metadata. This means tool names, descriptions, and schemas are reconstructed from function signatures and comments rather than explicit registration.
Enums not declared for constrained parameters. 'provider_preference' accepts only 'midtrans' or 'xendit' (per source code validation), but parameter schema does not declare enum constraints. LLMs will guess valid values instead of being guided to the correct options.
Output schema not documented. Checkout returns CheckoutResponse{CheckoutURL, ProviderTxID}, but the response structure is not visible in the tool definition, only inferred from the handler code. Downstream tools cannot know what fields to expect, breaking tool chaining.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 31 | - | v1 |
Error responses do not follow recovery-guide pattern. Code returns h.errorResponse(w, 'Invalid request body', http.StatusBadRequest) with no guidance on what to do next. LLMs receive bare HTTP status codes and generic messages ('Provider unavailable', 'Payment creation failed') with no actionable next steps.
Parameter constraints not documented in descriptions. Checkout has 'amount' (must be positive, max 999999999) and 'order_id' (max 50 chars), but descriptions lack explicit bounds or format specs. LLMs cannot infer from code validation; they need textual constraint declarations.
GetTransaction description incomplete. States 'currently returns pending status but placeholder for database query', this tells the LLM the tool is not ready for production. Placeholder text should not appear in live tool definitions.
Tool naming inconsistency. Checkout and HandleWebhook are noun-based; verb-noun convention (create_checkout, process_webhook) would be clearer per production baselines (90% of A+ tools start with action verb).
Webhook handler 'body' parameter typed as object with no schema. Source code extracts 'provider' from path but 'body' schema is undocumented. LLMs cannot validate webhook payload structure, risking malformed submissions.