Loading ToolBench
A licensing authority MCP server that gates autonomous AI agent actions through trust-based certification. Warrant consumes Splunk data via MCP and exposes tools for requesting action permission, checking licenses, reporting outcomes, and auditing the trust ledger.
Warrant exposes 5 well-named, domain-specific tools with clear descriptions (avg 180 chars, within baseline 194). All tools have input schemas with typed parameters and descriptions. Tool names follow verb_noun pattern (warrant_request_action, warrant_check_license, warrant_report_outcome, warrant_list_licenses, warrant_verify_ledger). Descriptions explain WHAT, WHEN, and WHY. However, output schemas are not formally documented in the source, responses are constructed ad-hoc (e.g., _license_payload returns a dict with no schema declaration). Error handling in warrant_report_outcome is present but minimal (returns {"recorded": False, "error": "..."}); no recovery guidance or error classification. Parameters use sensible defaults (agent_fingerprint="", confidence=0.7) and include constraints (confidence 0.0-1.0), but no explicit enum or range validation in schema. Missing: per-tool output schema documentation, structured error recovery guidance, and tool annotations (readOnlyHint/destructiveHint).
Return the current license (status, confidence, calibration, drift, probation strikes) for an action class, computed against the calling agent's fingerprint.
List every license Warrant currently holds, with the fingerprint in force.
Record the real-world outcome of an action so trust is updated. TRUST-BUT-VERIFY — two modes: MEASURED (preferred): pass `metric_url` (a JSON metrics endpoint), `metric` (key to read) and `upper_limit` (the committed forecast band). Warrant fetches the metric ITSELF and grades the outcome; the caller cannot lie its way to a license. SELF-REPORTED (fallback): pass `correct`. The outcome is recorded but permanently flagged evidence="self-reported" in the ledger, so an auditor can see exactly how much of a license rests on the agent's own word. Returns the license AFTER recording.
Ask Warrant for permission to perform an action autonomously. `agent_fingerprint` is the calling agent's identity (model id + prompt version, e.g. "gemini-2.5-flash:v3"). Licenses are pinned to the fingerprint that earned them — a different brain must re-certify. Returns a verdict: ALLOW (full autonomy), ALLOW_WITH_MONITORING (licensed by a thin margin — act, but page a human), or REQUIRE_APPROVAL (route to a human). `confidence` is the caller's stated confidence in this specific action (recorded later for calibration).
Output schemas not formally documented. Tools return dicts (e.g., _license_payload, warrant_report_outcome) but no JSON Schema or structured type hints visible in tool registration. LLMs cannot reliably parse response structure.
Error handling lacks recovery guidance. warrant_report_outcome returns {"recorded": False, "error": "..."} but does not classify errors as retryable/user-fixable/fatal or suggest next steps. LLM cannot self-correct.
Tool annotations missing. No readOnlyHint, destructiveHint, or idempotentHint declared. warrant_report_outcome is destructive (writes to ledger) but not marked as such. Agents cannot reason about side effects.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
Audit the trust ledger's tamper-evident hash chain. Every outcome is sha256-chained to the one before it; editing any historical record breaks the chain. Returns whether the chain is intact, plus how much of the record is independently measured vs self-reported — the two numbers an auditor of a licensing authority would ask for first.
Parameter constraints not enforced in schema. confidence parameter accepts 0.0-1.0 but no minValue/maxValue in JSON Schema; validation happens in code (max(0.0, min(1.0, confidence))). Schema should declare bounds.
warrant_report_outcome has complex conditional logic (measured vs self-reported mode) but parameter dependencies not documented. metric_url requires upper_limit, but this is only enforced at runtime with an error response, not declared in schema.