Loading ToolBench
Server provides 10 tools with consistent naming (verb_noun pattern: get_, list_, create_, etc.) and reasonable descriptions (avg ~120 chars). All tools have input schemas with type definitions and parameter descriptions. However, output schemas are not documented, LLMs cannot predict return structure. Parameter descriptions lack format constraints (e.g., max lengths stated in schema but not in description text). No error handling guidance or recovery paths. Missing tool annotations (readOnlyHint, destructiveHint) despite clear risk levels. Schemas are present but minimal; no pagination metadata documented for list_nodes despite offset/limit params.
Add a typed input or output port to a Katana node
Connect an output port from one node to an input port on another node
Create a new typed node in the nodegraph with optional name, parent, and position
Delete explicitly confirmed nodes from the nodegraph
Disconnect an output port from an input port
Get detailed information about a specific node, including ports and optionally parameters
Get the current status of the Katana host, including project file, dirty state, node count, node types, and timeline information
No output schemas documented. LLMs cannot predict return structure, forcing them to guess what fields are available for downstream tool calls. This breaks tool chaining and forces discovery detours.
Missing tool annotations (readOnlyHint, destructiveHint, idempotentHint). Tools are marked with Risk levels (READ_ONLY, WRITE, DESTRUCTIVE) in metadata but not exposed via MCP tool annotations. Agents cannot distinguish safe from dangerous operations.
No error handling guidance. Descriptions do not explain what errors can occur, when to retry, or how to recover. E.g., create_node does not document what happens if parent_name is invalid or node_type is unsupported.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
Inspect the current nodegraph structure, including root node, node count, selected nodes, and viewed node
List nodes in the nodegraph with optional filtering by type or name, with pagination support
Rename an existing node in the nodegraph
Parameter descriptions lack format constraints. Schema defines max lengths (128, 256 chars) and ranges (offset max 1000000, limit max 1000) but descriptions do not repeat these constraints. LLMs cannot read JSON Schema, they rely on description text.
Pagination metadata missing. list_nodes accepts offset/limit but does not document total count or next_cursor in return. LLMs cannot determine if more results exist or how to fetch them.