Loading ToolBench
Local-first, provider-agnostic agent that builds software. Exposes orchestration, memory, reasoning, skills, and file indexing tools via MCP server (stdio and HTTP transports).
Vishu has 17 well-named tools with clear descriptions and mostly complete schemas. Naming follows verb_noun convention (memory_write, file_search, dispatch). Descriptions are substantive (avg ~150 chars) and explain WHEN to use each tool. However, several tools lack parameter descriptions, output schemas are not documented, and error handling guidance is minimal. Tools like `orchestrate` and `dispatch` are ambitious but their complex behavior (subagent spawning, mode switching) is under-specified for LLM reasoning. No tool annotations (readOnlyHint/destructiveHint) despite clear read/write semantics.
Answer a question more reliably by sampling several candidates and selecting one by majority vote (default) or a judge model.
Build a bespoke sub-agent for a task (least-privilege tools ⊆ current tools, cited skills). Registration is gated; once approved the agent is routable by `dispatch`.
Route ONE task to the best executor — an approved bespoke agent (named in the task), a preset role, a synthesized agent, or a persona-mode switch (e.g. "interview me" → interviewer mode) — and run it. A too-vague task returns a clarifying question.
RAG-quote your own files: retrieve the top matching passages (wider excerpts than file_search) as grounded context to QUOTE from, each labelled with its source path. Answer the user strictly from these excerpts and cite the path; this keeps private files on the local model instead of guessing.
Index your local text files (VISHU_FILE_ROOTS or cwd) for search. Secrets are never indexed.
Search your indexed local files; returns the top matching paths + a snippet.
Output schemas not documented. Tools like `orchestrate`, `dispatch`, `solve`, and `run_workflow` return complex nested structures (diffs, agent results, ensemble outputs) but no schema is visible. LLMs cannot plan downstream calls or extract fields reliably.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint). Tools clearly marked as WRITE (memory_write, orchestrate, create_agent, dispatch, skill_run) lack destructiveHint. Tools marked READ_ONLY lack readOnlyHint. Annotations enable safer agent planning.
Error handling lacks recovery guidance. No visible error responses that tell LLMs what to do next (retry, ask user, try alternative tool). E.g., `dispatch` returning 'too-vague task returns a clarifying question' is mentioned in description but no error schema shown.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Fulfil a wishlist item: grant a system tool to a bespoke agent (gated). Only tools the system already exposes can be granted (add the tool first, then grant).
List the bespoke agents approved by the factory — name, granted tools, and each one's outstanding tool-wishlist.
Recall saved memories relevant to a query (hybrid search + link traversal).
Save a durable memory (a fact about the user/project). Pass `subject` to supersede a prior fact about the same thing.
Decompose a goal into distinct approaches, try each as an isolated subagent, run tests to prune failures, harvest the winner into the sandbox, and return the result with a diff to review. Landing it upstream (dev_commit/dev_push) stays gated.
Remove an approved bespoke agent so it is no longer routable. Revocation is gated.
Decide where a task should run BEFORE acting: which lane (brain=PA/ops, builder=engineering) and whether it fits the LOCAL small model (private/cheap/short) or needs the CLOUD (hard reasoning/coding/long). Under privacy-strict, personal-data tasks force local. Call this to keep private work off cloud prompts and cheap work off the premium quota.
Run a declarative JSON workflow: a DAG of tool nodes {id, tool, args?, needs?}. Nodes run in dependency order; a string arg may reference an upstream node's output with {{nodeId}}. Modes compose as nodes that call a mode-switch tool. Side-effecting classes (send/spend/delete/change_setting) and delegating tools are refused — run those through the normal gated loop.
Load or run a skill by name (returns its instructions, or its script output).
Search the skill library; returns matching skill names + one-line descriptions.
Answer a question with effort matched to its difficulty: a single call for trivial, self-consistency for medium, a multi-agent ensemble for hard. Use for any question where reliability matters.
Complex tool behavior under-specified. `orchestrate` (subagent spawning, branch testing, diff harvesting), `dispatch` (mode switching, persona routing), and `run_workflow` (DAG execution with node references) are powerful but lack detailed parameter constraints and output examples. LLMs may misuse them.
Parameter constraints missing. `best_of_n.n` (number of samples) has no min/max bounds. `run_workflow.spec` accepts 'JSON string' but no format/pattern constraint. `route_task.task` and `dispatch.task` accept free-form strings with no guidance on what constitutes a valid task.