Loading ToolBench
Standalone IOSM CLI with agent tooling, session management, and IOSM artifact orchestration
iosm-cli provides 7 tools with explicit schemas and descriptions, but quality is inconsistent. Tool names follow verb_noun convention (read, bash, edit, write, echo_session, rg, ask_user), which is good. However, descriptions vary widely in depth and actionability. The `read` tool has a solid description with security context (blocked paths). The `bash` tool lacks detail on timeout behavior and error handling. The `rg` tool documents truncation limits (2000 lines, 50KB), which is excellent. The `ask_user` tool has comprehensive parameter constraints (minLength, maxLength, minItems, maxItems) but descriptions are minimal. Most critically: no tool documents output schemas, no error recovery guidance is visible, and parameter descriptions lack format/constraint details that LLMs need. Average description length ~100 chars (below the 194-char baseline for A+ tools). No tool declares idempotency or destructiveness hints.
Ask the user a blocking product or architecture question and wait for the answer.
Execute bash commands in the current working directory
Echo a message with prefix: [session]
Edit a file by specifying line ranges and replacement content
Read the contents of a file with access logging. Some sensitive paths (.env, secrets, credentials) are blocked.
Search file contents using ripgrep. Output is truncated to 2000 lines or 50KB (whichever is hit first). If truncated, full output is saved to a temp file.
Write content to a file, creating it if it does not exist
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., what does `bash` return on success vs error? What fields does `rg` include in results?). This violates the response-shaper pattern.
Parameter descriptions lack actionable constraints. E.g., `bash` timeout parameter has no guidance on valid range (seconds? milliseconds? max value?). `rg` glob parameter says 'e.g. *.ts' but doesn't explain glob syntax or escaping rules. LLMs cannot validate inputs without explicit constraints.
No error recovery guidance. Tools like `bash` and `edit` can fail in many ways (permission denied, file not found, syntax error) but descriptions provide no hints on what to do next. Violates recovery-guide pattern.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
Destructive tools (bash, edit, write) lack confirmation or dry-run support. No tool annotations (destructiveHint) visible. An agent could accidentally delete files or run harmful commands without safeguards.
Tool composition unclear. The `read` tool blocks sensitive paths (.env, secrets, credentials) but no tool documents how to safely handle secrets or what alternatives exist. No guidance on when to use `bash` vs `rg` for searching.