Standalone IOSM CLI with agent tooling, session management, and IOSM artifact orchestration
Mixed quality across 6 tools. ask_user and tool_suggest have solid schemas and descriptions (80-90 chars), but tool_search, read, echo_session, and rg lack depth. ask_user is well-structured with timeout handling and custom answer support. tool_search and tool_suggest descriptions are generic (under 100 chars). read lacks parameter descriptions for offset/limit. echo_session is trivial (single string param). rg has good description but no input validation guidance. No tool annotations (readOnlyHint, destructiveHint). Error handling present but not recovery-focused. Schemas visible but descriptions inconsistent across tools.
Ask the user a blocking product or architecture question and wait for the answer.
Echo a message with prefix: [session]
Read the contents of a file with access logging. Some sensitive paths (.env, secrets, credentials) are blocked.
Search file contents using ripgrep. Output is truncated to 2000 lines or 51200 bytes (whichever is hit first). If truncated, full output is saved to a temp file.
Search available tools by name/description and return ranked matches with active/inactive status.
Suggest the best tools for a described task, ranking by task keywords and available tool metadata.
echo_session is trivial and lacks clear purpose. Single string parameter with no validation, no error handling, no output schema documented. Appears to be a demo tool unsuitable for production.
tool_search and tool_suggest descriptions are generic and under 100 chars. Missing context on when to use each vs the other, what structure they return, and how results are ranked. LLMs cannot distinguish between them.
read tool parameters (offset, limit) lack descriptions. Unclear if offset is 0-indexed or 1-indexed, whether limit is lines or bytes, or what happens if offset exceeds file length.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present. All tools marked READ_ONLY in metadata but not in schema. rg tool truncates output to temp file but no guidance on how to retrieve it.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Error handling lacks recovery guidance. ask_user returns 'cancelled' on timeout but no description of what the agent should do next. rg truncates output silently, no indication of temp file location or how to access full results.