Loading ToolBench
Self-hosted AI agent with persistent memory — CLI, Discord, and web API
Agent-os defines 9 tools with basic schemas and descriptions, but falls short of production quality. Tool names follow verb_noun convention (browser_navigate, task_create), which is good. However, descriptions are inconsistent in depth, some are 1-2 sentences (browser_click: 11 words), others more detailed (browser_navigate: 18 words). Most critically, parameter descriptions are sparse or missing entirely. For example, browser_click's 'ref' parameter lacks guidance on when to use accessibility nodeId vs CSS selector. Task management tools (task_create, task_update, subtask_update) have enum constraints for status, which is correct, but lack error recovery guidance. No tool documents output schemas, pagination, or what fields downstream tools should expect. Error handling is absent, no guidance on retryability, user-fixable vs fatal errors, or recovery paths. The server is STDIO-only, which is a hard transport limitation.
Click an element on the current page. Use the accessibility ref from browser_snapshot, or a CSS selector.
Close a browser session.
Navigate the browser to a URL and return the page accessibility tree. Creates a new session if none exists for this taskId.
Take a screenshot of the current page.
Get the current page accessibility tree and visible text content.
Type text into an input field on the current page.
Update the status of a specific subtask.
Parameter descriptions missing or minimal. browser_click 'ref' parameter lacks guidance on accessibility nodeId vs CSS selector precedence. browser_snapshot and browser_screenshot lack any description of what fields are returned.
No output schemas documented. Tools return data but LLMs cannot plan downstream calls without knowing field names and types. E.g., browser_navigate returns 'accessibility tree' but structure is undefined.
No error handling guidance. Tools lack recovery hints (e.g., 'If navigation fails, check URL format' or 'If task creation fails, verify subtask format'). No categorization of retryable vs fatal errors.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | <=2025-11-25 | v2 |
Create a new task and enter Planning Mode. This starts a workflow requiring user approval.
Update the status or title of a task.
Destructive operations lack confirmation or dry-run support. browser_close and task_update with status='failed' are irreversible but have no confirmation pattern.
taskId parameter uses string matching ('full or partial ID') without clear semantics. Ambiguous matching rules invite LLM errors and silent mismatches.