Loading ToolBench
An MCP proxy server and agentic hook tool that allows you to write and enforce policies on agent tool flows
Canopy MCP has only 2 tools with minimal schema definitions and sparse parameter documentation. Both tools have empty input schemas ({}), meaning no parameters are formally declared or typed. Descriptions exist but are generic and do not follow LLM-optimized patterns. No output schemas are documented. The tool `change_canopy_flow` relies on elicitation (user confirmation) but the elicitation flow is not formally documented in the tool schema. Error handling is present in the code but not surfaced in tool definitions. Security concerns exist: the tool uses middleware to filter based on policy, but the permission model is not declared in tool metadata.
Starts the process for changing/setting the active flow name for canopy. This will elicit user confirmation. This takes no parameters and you do not need prior information to run this.
Fetches the current configuration state of Canopy.
Both tools have empty input schemas (Input: {}). No parameters are formally defined with types or constraints, violating the core requirement that every parameter must have a description and type declaration. Score HARD-CAPPED at 0 for schema.
Tool description for change_canopy_flow mentions 'elicit user confirmation' and says 'This takes no parameters', but elicitation semantics are not formally documented in the tool schema. The response type (str) and error cases are implicit in code, not declared in metadata. LLMs cannot plan this interaction without explicit schema hints.
No output schemas documented for either tool. get_canopy_status returns a dict with 'path' and 'picked_flow' fields, but this structure is not declared. change_canopy_flow returns {'status': 'Updated successfully.'} or raises an exception, but no return type schema exists. LLMs cannot plan downstream calls without knowing return structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | 2026-07-28+ | v2 |
Tool descriptions are under 200 chars and generic. 'Fetches the current configuration state of Canopy' lacks detail on what 'configuration state' means and when to call this tool. 'Starts the process for changing/setting the active flow name' does not explain failure modes or when the user should invoke this vs other tools.
No error handling guidance in tool definitions. change_canopy_flow can fail with 'Flow change was not accepted by user' or 'Audit webhook failed', but these error cases are not declared. LLMs cannot plan recovery (e.g., retry, ask user for input, escalate).
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). get_canopy_status is marked READ_ONLY in the server config but this is not declared in the tool metadata. change_canopy_flow is marked WRITE but has no annotation. LLMs cannot infer idempotency or side effects from the schema alone.
Permission model not declared. The PolicyMiddleware filters tool access based on policy.is_allowed(), but this permission check is not exposed in tool metadata. LLMs cannot discover which tools are available without attempting calls. No scope declarations (e.g., 'read:policy', 'write:flow').