Loading ToolBench
Torii defines 2 tools with clear schemas and structured output. torii_policy has a well-documented read-only operation with proper parameter validation. dynamic_provider_tools is a meta-tool that dynamically wraps provider CLIs. Both tools have descriptions (194 and 156 chars respectively, within baseline 34 - 392 range). Schemas are properly typed with required fields and descriptions. However, naming lacks action verbs (torii_policy should be get_policy or read_policy; dynamic_provider_tools is generic). Error handling is present but minimal, no recovery guidance or actionable error messages. Output schemas are structured but lack documented field descriptions. No tool annotations (readOnlyHint, destructiveHint) despite clear risk profiles.
Dynamically registered provider CLI tools. Each installed provider becomes an MCP tool with the provider's configured tool name. Tools accept target (for target-aware providers) and args (array of command-line arguments to pass to the provider executable).
Read the active accept and deny rules for an installed Torii provider and optional target. This never executes a provider, reads credentials, or changes policy.
Tool names lack action verbs. 'torii_policy' should be 'get_policy' or 'read_policy'; 'dynamic_provider_tools' is generic and does not convey the action (invoke, execute, run). LLMs infer intent from verb-noun naming.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite explicit risk labels. torii_policy is READ_ONLY; dynamic_provider_tools is IRREVERSIBLE. Annotations enable LLM safety reasoning and are part of current MCP spec (2026-07-28).
Error messages lack recovery guidance. Code returns McpError::invalid_params with raw error text (e.g. 'expected { provider: string, target?: string }') but does not suggest next steps or list available providers. Pattern: recovery-guide.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2025-06-18+ | v2 |
Output schema for torii_policy (PolicySnapshot) is structured but fields lack descriptions in the MCP tool definition. LLMs cannot infer what 'ignored_accept', 'minimum_accept_tokens', or 'unmatched' mean without inline documentation.
dynamic_provider_tools is a catch-all meta-tool that dynamically wraps provider CLIs. This violates single-responsibility principle (pattern:tool). Each provider should be a distinct, named tool with its own schema and description, not a generic 'args' array.