Python SDK with optional HTTP API for agent input scanning, tool-call guardrails, and audit logs
Four tools with basic descriptions but critical schema gaps. All tools lack explicit input parameter type definitions in visible code. Descriptions are present (50-150 chars) but generic, they explain the policy demo rather than guiding LLM tool selection. No output schemas documented. No error handling guidance. Tools are well-named (verb_noun pattern) but parameter descriptions are minimal. Security-focused domain is clear, but tool definitions do not meet production quality baselines.
Send a webhook (simulated). URL-based egress policy blocks destinations outside the domain allowlist. This is the clearest blocking demo for external network calls.
Read a file (simulated). Only paths inside file_path_allowlist are allowed. Sensitive paths such as /etc/passwd are blocked.
Search data (simulated). Search results are automatically redacted when they contain PII such as emails or API keys. With pii_action=block they are replaced with labels such as [EMAIL] and [API_KEY].
Send an email (simulated). Recipient domains are inspected by egress policy. In this demo, external email destinations produce a warning and remain auditable.
Input schemas lack explicit type definitions. Code shows parameter names and descriptions in docstrings, but no JSON Schema with 'type' fields visible.
No output schemas documented. Tools return strings but LLMs cannot infer structure of results (e.g., does search() return a list or a single string? What fields does it contain?). Blocks downstream tool chaining.
Descriptions are policy-demo-focused, not LLM-selection-focused. E.g., 'Send a webhook (simulated). URL-based egress policy blocks destinations outside the domain allowlist.' explains the guard behavior, not when/why the LLM should call this tool vs. alternatives. Missing: what does it return? What are prerequisites?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
No error handling guidance. Tools do not document what errors are possible, whether they are retryable, or what the LLM should do if a call fails. E.g., read_file blocks /etc/passwd, but the error response is not shown.
Parameter descriptions are minimal. 'url' → 'The webhook URL destination' does not explain format, constraints, or examples. 'query' → 'Search query string' does not hint at syntax, length limits, or what fields are searched.