Loading ToolBench
A multi-server MCP control plane demonstration featuring ContextForge governance, OPA policy enforcement, and multiple specialized MCP servers (controlplane, corpus, erp-payments, expense-db, fx-rates, notify, policy-docs, sales-tax) plus A2A agents (auditor, chair, payments, room).
13 tools with mixed quality. Naming is verb-forward and clear (register_mcp_server, list_control_plane, wire, approve, reimburse). Descriptions are present and contextual (avg ~120 chars), explaining WHAT and WHEN. Input schemas are visible with types and descriptions for all parameters. However, output schemas are NOT documented, LLMs cannot predict return structure. Error handling is absent; no recovery guidance or categorization. Parameters lack constraints (enums, ranges, format specs). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels (DESTRUCTIVE wire, READ_ONLY list_control_plane). Composition is sound, tools are single-purpose and chainable (e.g., list_pending_expenses → get_expense → approve → reimburse). Security risk: wire tool accepts approval flag as parameter rather than enforcing server-side policy (though description notes OPA enforcement, the tool itself is not defensive).
Mark an expense as approved for reimbursement.
Ask the OPA policy engine directly whether a payment of `amount` (with or without `approval`) WOULD be allowed — interrogating the policy live, without actually attempting the payment. Returns allow + any deny reasons.
Return the private note the given owner contributed (empty string if none).
Fetch a single expense by its id.
Return the demo FX rate to convert 1 unit of `base` into `quote`.
Fetch the raw text of a receipt by its receipt id (e.g. 'rcpt_pii').
No output schemas documented. LLMs cannot predict return structure (fields, types, nesting). Forces agents to guess what data is available for downstream tool calls.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Tools marked with Risk levels (READ_ONLY, WRITE, DESTRUCTIVE) in metadata but not exposed to LLM via MCP annotations. LLM cannot reason about safety without explicit hints.
No error handling or recovery guidance. Tools lack descriptions of failure modes, retryability, or next steps. E.g., wire tool does not document what happens if OPA denies; approve does not explain what error occurs if expense_id is invalid.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 65 | 2026-07-28+ | v2 |
List everything ContextForge is governing: federated MCP servers, A2A agents, and the virtual servers (each a least-privilege tool scope).
List the supported quote currencies.
Return all expenses currently awaiting approval.
The audit trail of recent tool invocations the control plane BLOCKED (policy violations / errors), newest first — what was stopped and when.
Register a NEW MCP server into the ContextForge gateway so its tools join the governed catalog. `url` is the server's MCP endpoint, reachable from the gateway (e.g. 'http://my-service:8000/mcp').
Reimburse an approved expense (low-risk).
Wire funds to a payee. High-risk: large wires require approval=true (enforced by the gateway's OPA policy, not here).
Parameters lack constraints (enums, ranges, format specs). E.g., amount in wire/reimburse has no min/max; approval is boolean but no guidance on when true is required; currency codes in get_fx_rate have no enum or pattern.
list_* tools lack pagination parameters (limit, offset, cursor). No indication of result count limits. Large result sets could exhaust context window.