Loading ToolBench
Strong foundation with well-structured schemas and comprehensive descriptions. All 8 tools have explicit input schemas with type definitions and parameter descriptions. Tool names follow verb_noun convention (create_envelope, check_envelope_status, list_envelopes, send_reminder, seal_envelope, fund_wallet). Descriptions are detailed (150-300 chars) and explain WHAT, WHEN, and consequences. Tool annotations (destructiveHint, readOnlyHint) are present. Key gaps: no output schemas documented; error handling lacks recovery guidance; some parameter constraints could be more explicit (e.g., amount_wei format); no batch variants despite repetitive patterns.
Check a signing envelope: per-signer signing status and times, plus each signer's delivery_status (pending / delivered / undeliverable — whether the signing-request email reached them, distinct from whether they signed). Accepts EITHER the ambient KYSIGNED_AUTHORIZATION creator key OR a per-envelope tracking_token (ktt_…, returned by every create — F-30.7): the token needs NO API key and reads exactly its own envelope, so the no-key wallet path polls through the same tool it created with. An explicit tracking_token wins over the ambient key.
Check a signing envelope without an API key, using a per-envelope tracking_token (ktt_…). Reads exactly its own envelope. The no-key wallet path polls through the same tool it created with.
Create a new signing envelope. Sends a PDF to one or more signers (max 20); at completion every party receives a sealed evidence-bundle PDF. Provide EXACTLY ONE of pdf_base64 (≤ ~3 MB) or pdf_url (any size, fetched server-side). Consumes a creator credit and emails each signer.
Create a signing envelope without an API key, using a funded wallet (F-30 / AC-170). Requires prior wallet funding via fund_wallet. Sends a PDF to one or more signers (max 20); at completion every party receives a sealed evidence-bundle PDF. Provide EXACTLY ONE of pdf_base64 (≤ ~3 MB) or pdf_url (any size, fetched server-side). Consumes wallet balance and emails each signer.
Output schemas not documented. LLMs cannot infer what fields to expect from tool responses (e.g., create_envelope returns envelope_id, tracking_token, callback_secret, not visible in schema). This forces agents to guess downstream field names and risks broken tool chains.
Error handling lacks recovery guidance. Tools return coded isError results but descriptions do not explain what to do on failure (e.g., 'If envelope not found, try list_envelopes()' or 'Retry with a different tracking_token'). Agents cannot self-correct.
Duplicate tool pairs (create_envelope + create_envelope_no_key, check_envelope_status + check_envelope_status_no_key) create naming ambiguity. LLMs must reason about which variant to use based on wallet state. Consider a single tool with a 'use_wallet' boolean parameter or clearer naming (e.g., create_envelope_with_api_key vs create_envelope_with_wallet).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2026-07-28+ | v2 |
Fund a wallet for pay-per-envelope signing (F-30 / AC-170). Requires an injected payment signer (configurePaymentSigner) or KYSIGNED_RUN402_ALLOWANCE_PATH. Returns a signed transaction ready to broadcast; the caller is responsible for submitting it to the blockchain. Wallet funding is a prerequisite for the no-key wallet path (create_envelope_no_key / check_envelope_status_no_key).
List the envelopes you (the authenticated creator) have sent, with each one's status.
Manually seal and distribute an envelope (Phase 10 — F-9.1). Only valid when auto_close=false at creation; sealing triggers the completion-distribution pipeline (evidence bundle assembly + email to all parties). Idempotent: calling seal on an already-sealed envelope succeeds silently.
Send a reminder email to all pending signers on an envelope.
Parameter constraints not fully explicit. 'amount_wei' is described as 'string for precision' but no min/max bounds, format pattern, or example given. 'expiry_days' lacks upper bound. LLMs may pass invalid values (negative, zero, absurdly large).
list_envelopes has no pagination parameters (limit, offset, cursor). If a creator has hundreds of envelopes, the tool returns all at once, bloating context and risking token exhaustion. No documented result limit or next_cursor field.