Loading ToolBench
Self-hosted, caller-configured default-deny action registry + append-only spend ledger + human sign-off audit trail, exposed as MCP tools. classify() is unconditional GATED -- that invariant is never configurable. Governance/safety flagship: stop an agent from doing something irreversible without a human noticing.
Six tools with complete schemas and strong descriptions. All tools have clear verb-noun naming (classify_, is_, get_, record_, evaluate_) and detailed descriptions (100-250 chars) explaining purpose, invariants, and constraints. Input schemas are fully typed with parameter descriptions. However, output schemas are not documented, callers must infer response structure from descriptions. Error handling is implicit (routes normalize to {ok: bool, ...}) but not explicitly described in tool definitions. No tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications in metadata.
Default-deny classification for an action_type: ALWAYS returns GATED, whether or not the action_type is registered. There is no fail-open branch and no argument that can change this -- it is the one invariant this server refuses to make configurable. Use is_action_registered to check whether an action_type has a known enforcement entry; that is a separate, informational question.
Sum spend_intent amounts recorded within the last window_days and compare against limit_usd. Never raises -- an empty ledger reads as total_spent_usd=0.0, within_budget=True.
12-hex sha256 digest of the currently loaded action registry, plus its entry count. Changes iff the registry's contents change -- this is the value a human sign-off (rails-mcp sign, CLI-only, never an MCP tool) records into the audit ledger.
Current active human sign-off of the registry (operator, signedAt, the registryHash they signed, optional note), or null if never signed or most-recently revoked. Read-only -- signing itself is a CLI-only ceremony (`rails-mcp sign --operator ...`), never something reachable from this or any MCP tool call.
Whether action_type has an entry in the loaded rails registry, plus its enforcement_layer / enforcement_pointer / ceremony when present. Registration is informational only -- classify_action's GATED verdict never depends on it.
Output schemas not documented. Tool descriptions explain what is returned (e.g., 'returns GATED', 'returns ok: bool, ...') but formal response schemas are absent. LLMs cannot plan downstream operations without knowing response structure.
Tool annotations missing. classify_action and record_spend_intent have explicit risk classifications (READ_ONLY, WRITE) in metadata but are not exposed as readOnlyHint/destructiveHint in MCP tool definitions. Clients cannot infer safety properties.
Error handling not explicit in tool definitions. routes.py normalizes exceptions to {ok: bool, ...} but tool descriptions do not document error cases, recovery guidance, or what 'ok: false' means. LLMs cannot self-correct on failure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | 2026-07-28+ | v2 |
Append one spend-intent record to the append-only ledger. This RECORDS intent to spend -- it never calls a vendor, a paid API, or a broker, and it cannot itself authorize or block a spend.
Parameter descriptions lack format/constraint details. evaluate_budget's window_days defaults to 30.0 but no min/max bounds are stated. record_spend_intent's amount_usd has no range constraint. LLMs may pass invalid values.
No pagination or result limits documented. If get_signoff_state or is_action_registered return large nested structures, no guidance on truncation or streaming. Baseline pattern expects pagination for list-like tools.