Nova exposes 5 tools with significant definition gaps. Tool names lack clear action verbs (ask_user, delegate_to_agent are vague; code_run is generic). Descriptions are present but minimal (10-50 chars), below the 50-200 char LLM-optimized baseline. Input schemas are partially visible in frontend code but lack complete type definitions and parameter descriptions in the backend. The code_run tool accepts arbitrary Python with no validation or safety constraints. edit and write tools lack content parameter schemas. No output schemas documented. Error handling is absent, no recovery guidance or categorization.
Request user input through a wizard-style interface with support for text, select, confirm, and textarea input types
Execute Python code and return the result
Delegate a task to another agent
Edit an existing file with the provided content
Write content to a file
code_run tool accepts arbitrary Python code with no input validation, sandboxing, or safety constraints. Executing untrusted LLM-generated code is a critical security vulnerability.
edit and write tools lack documented content parameter schemas. Input schema is incomplete, no type, description, or size limits visible in source.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields without knowing response structure.
Tool descriptions are 10-50 characters, well below the 50-200 char LLM-optimized baseline. Descriptions lack WHAT/WHEN/WHY context needed for tool selection.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 37 | 2026-07-28+ | v2 |
Tool names lack clear action verbs. 'ask_user' is vague (ask for what?), 'code_run' is generic, 'delegate_to_agent' is ambiguous. LLMs cannot infer intent from names alone.