Loading ToolBench
Governance layer for healthcare AI agents over MCP: identity, permission, tamper-evident audit, recall, and a PHI guard. By Haver AI.
Well-structured healthcare governance tools with clear naming (verb_noun pattern), comprehensive descriptions (avg 180 chars), and complete input schemas using Zod. All 7 tools have descriptions and typed parameters. Output is consistently structured JSON. However, output schemas are not formally documented for LLM consumption, error handling lacks recovery guidance, and some parameter descriptions could be more prescriptive about constraints and formats. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels. Composition is sound, each tool has one responsibility and tools chain via agent_id.
Decide whether a registered agent may perform an action on a resource, based on its scopes and status. Recalled or unknown agents are always denied. The decision is written to the audit log.
Return recent audit entries, optionally filtered by agent or resource.
Revoke an agent so all future permission checks deny it. Use when an agent misbehaves or is decommissioned. The recall is audited.
Append a tamper-evident audit record for something an agent did. Entries are hash-chained, so any later modification is detectable.
Register an AI agent with a stable identity, owner, role, and permission scopes. Scopes use the form 'action:resource' (e.g. 'read:labs'), and '*' wildcards are allowed.
Heuristically scan text for common protected health information (SSN, MRN, email, phone, dates) and return findings plus a redacted copy. A guardrail, not a certified de-identification tool.
Output schemas not formally documented. Tools return JSON via asText() helper, but LLMs cannot see the response structure (fields, types, required vs optional). This forces LLMs to infer output shape and risks misinterpreting audit entries, permission decisions, or PHI findings.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. recall_agent is IRREVERSIBLE, register_agent and record_action are WRITE operations. LLMs cannot distinguish safe reads from destructive writes without annotations.
Error handling lacks recovery guidance. recall_agent returns {ok: false, error: 'unknown agent'} but does not suggest next steps (e.g., 'Try get_audit_trail to find valid agent IDs'). check_permission returns deny/allow but no guidance on how to escalate or request new scopes.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
Recompute the audit hash chain and report whether the log is intact or has been tampered with.
get_audit_trail limit parameter lacks explicit constraint documentation in description. Schema declares max(500) but description does not state 'Maximum 500 entries per call' or explain why. LLMs may not respect the constraint.
scan_phi description calls it 'a guardrail, not a certified de-identification tool' but does not specify what patterns it detects (SSN, MRN, email, phone, dates mentioned in description but not in parameter docs). LLMs may over-rely on it for compliance.