Loading ToolBench
MCP policy proxy for AI agents: hard spend limits, destructive-action gating with approvals, a kill switch, scoped credentials, dry-run writes with mutation diffs, loop breaker, blast-radius caps and a hash-chained audit log. No LLM calls, no phone-home, no account.
agentguard exposes 10 CRM tools with complete input schemas and descriptions. Naming follows verb_noun convention (crm_list_contacts, crm_create_contact, crm_delete_contact). Descriptions are present and actionable (avg ~80 chars), explaining what each tool does and its risk level. However, output schemas are not documented in the visible source, the tool definitions show input schemas but no explicit return type documentation. Parameter descriptions are adequate but lack constraint details (e.g., email format, name length limits). Error handling is mentioned for some tools (NOT_FOUND, UPSTREAM_ERROR) but recovery guidance is minimal. The crm_frobnicate tool intentionally lacks annotations to test unknown handling, and crm_fail is designed for error-path testing, both reduce overall quality. Security annotations (READ_ONLY, WRITE, DESTRUCTIVE) are present and well-applied, which is a strength.
Charge a customer's card on file (amount in cents). Returns the charge id and amount_usd.
Create a contact. Returns the new id and created_at timestamp.
Permanently delete a contact. Irreversible. Returns { deleted, id }.
Always returns a structured UPSTREAM_ERROR result, for error-path tests.
A tool with no annotations and no recognizable verb — exists to test `unknown` handling.
Fetch one contact by id. Read-only; NOT_FOUND when the id does not exist.
List every contact in the CRM. Read-only; returns id, name, email and updated_at.
Output schemas not documented. Tool definitions show input schemas but no explicit return type documentation visible in source. LLMs cannot plan downstream calls or extract required fields without knowing what each tool returns.
crm_update_contact 'fields' parameter is an untyped object with no schema constraints. Description says 'name and/or email, both optional' but lacks formal validation rules. LLMs may pass invalid field names.
crm_frobnicate has no recognizable verb and minimal description ('exists to test unknown handling'). This tool is intentionally poorly named and documented, reducing overall quality score.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2025-06-18+ | v2 |
Search contacts by name or email substring. Returns matching contacts (no annotations on purpose).
Send an email to a contact through the CRM. Returns the message id and sent_at.
Update fields on a contact. Returns the id and updated_at; NOT_FOUND for unknown ids.
Parameter descriptions lack format constraints. E.g., crm_create_contact 'email' parameter has no format validation hint (should specify RFC 5322 or similar). crm_charge_card 'amount_cents' lacks min/max bounds.
Error recovery guidance is minimal. Tools mention NOT_FOUND and UPSTREAM_ERROR but do not guide LLM on next steps (e.g., 'Try search_contacts() with a partial name').