Loading ToolBench
A Dify plugin that integrates Signet authentication for agent identity management and cryptographic signing of agent actions
Three tools with basic schemas and descriptions, but significant gaps in parameter documentation, output schema clarity, and error handling guidance. Tool names follow verb_noun convention (audit_query, sign_action, verify_receipt), which is good. However, parameter descriptions are minimal (e.g., 'JSON string of tool parameters' lacks format/constraint details), and output schemas are not formally documented, responses are inferred from code inspection only. No error recovery guidance, no permission declarations, and no input validation messaging. The server uses dify-plugin framework (transport unknown, likely STDIO-only based on main.py structure), which caps protocol readiness severely.
Query the Signet audit trail
Sign an agent action with Ed25519
Verify a Signet receipt
Output schemas not formally documented. Tools return JSON strings via create_text_message(), but response structure (fields, types, required keys) is not declared in tool metadata. LLMs cannot plan downstream calls or extract fields reliably.
Parameter descriptions lack format constraints and validation rules. 'JSON string of tool parameters' (SignActionTool.params_json) does not specify valid JSON structure, required keys, or error handling for malformed input. 'Time period to query' (AuditQueryTool.since) lacks examples of valid formats (24h, 7d, etc.).
No error recovery guidance. Exception handling in verify_receipt.py catches all errors and returns {valid: false, error: str(e)}, but does not guide the LLM on next steps (retry, check input format, contact admin). SignActionTool silently converts invalid JSON to {raw: params_json} without warning.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
No permission or scope declarations. Tools interact with cryptographic signing and audit trails but do not declare required permissions (e.g., 'sign:action', 'read:audit'). Agents cannot be configured with least-privilege access.
Credentials passed via runtime.credentials (provider/signet.py) but no documentation of required credential structure (signet_key_name, format, how to obtain). LLMs cannot infer setup requirements.