Loading ToolBench
The guardian layer for AI agents — identity, secrets, audit, proxy enforcement via MCP
Haldir demonstrates strong governance-domain fundamentals with clear naming, structured schemas, and well-documented parameters. All 15 tools follow verb_noun conventions (haldir_create_session, haldir_get_secret, haldir_log_audit_action). Input schemas are complete with proper types and descriptions for ~95% of parameters. Descriptions are detailed (average 180 chars) and explain WHEN/WHY to use each tool. However, tool annotations (readOnlyHint, destructiveHint) are absent, critical output schemas are not formally documented, and error recovery guidance is minimal. The audit/cryptographic tool cluster (tools 8-15) has excellent domain-specific descriptions but lacks field-level documentation for returned evidence structures.
Check whether a session is authorized for a given scope. Returns {allowed: bool} — call before a risky tool to gate it.
RFC 6962 consistency proof — prove that a tree only grew, never shrank or reordered.
Create a scoped Haldir session for an AI agent. Returns a session_id the agent uses for every subsequent tool call. Use spend_limit to cap the agent's total spend; scopes (e.g. ['stripe:refund', 'postgres:read']) gate which secrets and approvals it can access.
Export a tamper-evident evidence pack: audit entries, tree head, inclusion/consistency proofs, and a signature. Everything needed to independently verify the audit trail.
Retrieve a secret by name. Requires a session_id that holds the secret's scope_required. Returns {value: ...}.
Get a session's current state — remaining budget, scopes, TTL, validity.
Output schemas not formally documented. Tools like haldir_create_session, haldir_get_session, haldir_evidence_pack return complex nested structures (session objects, Merkle proofs, evidence packs) but LLMs cannot see field types, required fields, or nested object structure. This forces LLMs to guess at response shape.
Tool annotations missing (readOnlyHint, destructiveHint). Tools like haldir_revoke_session and haldir_store_secret perform irreversible state changes but are not marked with destructiveHint. READ_ONLY and WRITE risk labels exist in source but are not exposed in MCP tool definitions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 67 | 2026-07-28+ | v2 |
Spend summary for an agent or session: total USD and a breakdown by tool. The question a budget owner asks.
RFC 6962 inclusion proof — prove that a leaf (audit entry) exists at a given index in the Merkle tree.
List all secret names in this tenant's vault (values never leak).
Log a tool call to the audit trail. The entry is SHA-256 hash-chained and leaf-hashed into the tenant's RFC 6962 Merkle tree so it's retroactively provable.
Query the audit trail. Filter by agent_id, session_id, tool, flagged.
Revoke a session immediately. Idempotent; returns 404 if already gone.
Store an encrypted secret (API key, token, credential) in the Vault. AES-256-GCM with AAD binding. scope_required gates which sessions can later read it.
Get the current signed tree head for this tenant's audit Merkle tree. Use this to verify chain proofs.
Prove that an audit entry exists in the Merkle tree and has never been tampered with. Returns a hash chain and the tree head that proves inclusion.
Error recovery guidance absent. Tool descriptions lack actionable next steps when operations fail. E.g., haldir_check_permission returns {allowed: bool} but no guidance on what to do when allowed=false, or haldir_get_secret returns no recovery hint if scope_required check fails.
Cryptographic tool descriptions lack field-level precision. haldir_verify_chain, haldir_inclusion_proof, haldir_consistency_proof, haldir_evidence_pack reference RFC 6962 and Merkle trees but do not document the exact fields in returned 'hash chain', 'proof', or 'signature' objects that LLMs must extract and pass downstream.
haldir_list_secrets has empty input schema (no required or optional params). Description does not explain filtering, ordering, pagination, or result size limits. Agents cannot tell if this returns 5 secrets or 500.
haldir_get_spend description is vague ('Spend summary for an agent or session'). Does not clarify what 'breakdown by tool' contains, whether it includes pending/estimated costs, or how child sessions aggregate to parent agents.