Loading ToolBench
The missing retry loop guard for Claude Code — plus vision/DOM compression and Japanese input helpers via MCP
helix-agent has 12 tools with schemas and descriptions present, but quality is inconsistent. Descriptions are present but often generic or task-focused rather than LLM-optimized. No output schemas documented. Error handling is minimal. Tool naming is reasonable (verb-noun style) but some tools combine multiple concerns (e.g., 'think' and 'agent_task' both delegate reasoning). File tools are well-guarded but lack pagination. Memory tools lack filtering documentation. No tool annotations (readOnlyHint/destructiveHint). Overall: below production baseline.
Add a memory to shared Qdrant storage.
Run a multi-step delegated task with provider-specific behavior. Ollama and OpenAI-compatible providers use the built-in ReAct loop. Codex runs as an autonomous implementation/review agent.
Safely evaluate a math expression.
Fork a sub-task with context inheritance (Claude Code forkSubagent pattern).
List files in a directory.
Run multiple tasks in parallel on local LLMs. Auto-selects optimal model per task. Tasks using e2b/e4b run truly in parallel. Tasks requiring 31b run sequentially (31b parallel degrades quality due to GPU contention).
Read a file's contents.
Parameter descriptions missing or minimal across most tools. Rubric requires every parameter to have a description explaining what it controls. 'model', 'provider', 'mode', 'effort', 'timeout' appear in multiple tools with no descriptions visible in schema.
No output schemas documented for any tool. Rubric requires documenting return types so LLMs know what fields to expect. Agents cannot plan downstream calls without knowing response structure.
Tool 'think' and 'agent_task' both delegate reasoning tasks with nearly identical parameters. Violates single-responsibility principle. LLM must reason about which to use. Consider consolidating or clarifying distinct use cases.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | <=2025-11-25 | v2 |
Run a shell command (restricted to safe commands).
Search for a pattern in a file. Input: JSON with 'path' and 'pattern' keys.
Search shared memory (Qdrant) for relevant information.
Delegate a reasoning, analysis, or code task to a selected provider.
Write content to a file. Input: JSON with 'path' and 'content' keys.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present. Rubric rewards these for spec alignment. Tools like 'write_file', 'add_memory', 'run_command' should be marked destructive; 'read_file', 'search_memory' should be marked read-only.
Error handling is minimal. Tools return plain error strings with no recovery guidance. Rubric requires: 'Error responses must tell the LLM what to do next.' E.g., 'File not found' should suggest 'Try list_files() to see available files.'