Loading ToolBench
A multi-agent orchestrator desktop app that coordinates AI agents with tools for HTTP APIs, web search, folder access, knowledge bases, and user interaction
OpenOrbit provides 14 tools with mostly complete descriptions (avg 156 chars) and clear naming conventions. All tools start with action verbs (web_search, fetch_web_content, list_*, read_*, save_*, get_*, delete_*, ask_user, write_checklist, find_skill). Input schemas are present with type definitions and descriptions for most parameters. However, several tools lack output schema documentation, error handling guidance is minimal, and some parameter constraints are missing. The ask_user tool has complex nested field objects without full schema visibility. Tool composition is strong (clear chains: list_granted_folders → list_folder_contents → read_folder_file), and security considerations are evident (no credentials exposed, permission-gated folder access). Main gaps: no documented return types, limited error recovery guidance, and missing parameter constraints (e.g., minLength/maxLength on some string params).
Ask the user a single question and wait for their real answer — this is a tool call, never text you write in your reply. Never bundle more than one question into a single call; if you need several answers, call this again after each one resolves. Use field.type "single_select" with a short list of real options when the answer is naturally a pick-one choice (the UI adds its own free-text escape hatch automatically); use "text" for anything open-ended. Set field.required=false with a sensible field.placeholder whenever skipping is reasonable — the placeholder becomes the literal answer if skipped or timed out. Set rememberAsUserInfo=true only for facts worth every agent knowing later (the same bar as save_user_info). The answer can come back as "[no answer — timed out]" or "[user cancelled — did not answer]" — neither is real data: don't treat either as the user's actual answer. Report that you couldn't get it and continue without guessing, or ask again differently if that's more useful than dropping it.
Delete one piece of your own data by key.
Fetch a specific URL (e.g. one returned by web_search) and extract its readable text content. Use this to read the actual content of a page rather than relying on the search snippet alone.
Search GitHub for up to 3 real, purpose-built skill files (SKILL.md) matching a short domain query (e.g. 'budget management', 'vedic astrology'), and get back a merged, deduplicated set of key domain-knowledge points to fold into the new agent's prompt draft. Default to calling this before drafting any agent with a named domain, hobby, profession, or specialized knowledge area — even one you already know about from training, since a real purpose-built reference beats drafting from memory alone. Only skip it for a genuinely generic agent with no real subject-matter domain.
No documented output schemas for any tool. LLMs cannot plan downstream calls or extract return values without knowing field names and types.
ask_user tool has nested 'field' object parameter with type 'object' but no schema for its properties (type, required, placeholder). LLM cannot validate field structure.
Error handling guidance is absent. Tools do not document what errors can occur, whether they are retryable, or what the LLM should do next (e.g., 'file not found, try list_folder_contents first').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
Retrieve one piece of your own data by key, previously saved with save_agent_data.
List every key/value pair you've saved for yourself with save_agent_data.
List the files and subfolders inside a path. The path must be one of the granted folders (see list_granted_folders) or a subdirectory of one — any other path is denied.
List the root folders the user has explicitly granted access to via the Folders widget. Use this first to discover what's available before listing or reading files.
List everything in the knowledge base (title, path, category, when last synced, and kind). Optionally filter by a partial, case-insensitive category match. Use this to discover what reference material is available before reading. Each entry's `kind` says how to open it: 'file' and 'url' entries are read with read_knowledgebase_file, while a 'folder' entry is a local directory the user attached — browse it with list_folder_contents and read what's inside with read_folder_file. One call covers both, so there is no need to call list_granted_folders as well.
Read the content of a file — plain text as well as PDF, Word (.doc/.docx), Excel (.xls/.xlsx), PowerPoint (.pptx), HTML, and OpenDocument (.odt/.ods/.odp) files, parsed into plain text. The file must be inside one of the granted folders (see list_granted_folders) — any other path is denied.
Read the stored content of a knowledge-base file by its id. Content is capped at 100 KB; the text ends with '[...truncated]' if the original was longer. Use list_knowledgebase_files first to find the id.
Save a piece of your own data under a key, so you can retrieve it later in this or a future conversation. Overwrites any existing value for that key. Only you can read this data back — other agents cannot see it.
Search the public web for a short query (2-6 words work best) and get back a list of results (title, url, description). Use fetch_web_content afterward to read a specific result's full page text. Run multiple short, varied searches rather than one long query.
Report your plan for handling this request, and update it as you go. Call this before doing anything else — even a one-item checklist for a direct answer that needs no tools — then call it again whenever your plan changes (e.g. a branch resolves, a step starts, a step finishes) or before your final reply, so every item ends as 'completed' or 'cancelled' rather than left 'pending'. Always send the FULL current checklist — this replaces what you last wrote, it does not append to it.
web_search and fetch_web_content lack result limits in descriptions. No guidance on pagination, max results, or token cost implications for large responses.
read_folder_file and read_knowledgebase_file mention truncation at 100 KB but do not document how to handle truncated content or request continuation.