Loading ToolBench
A multi-bot coordination server with MCP integration, skill library management, and bot creation/configuration tools. Supports multiple IM channels (Slack, Feishu, etc.), integrations via MCP servers, and autonomous bot orchestration.
18 tools with schemas present but significant quality gaps. Tool names are action-oriented (create_, configure, connect_, disconnect_, add_, grant, remove_, build, equip, harvest, channel_check, ask_login, avatar), good verb-first convention. However, descriptions vary widely in quality: some are detailed (create_bot, create_group), others are vague or overly technical (configure, harvest). Parameter descriptions exist but many lack type constraints, format guidance, or validation rules. Output schemas are not documented anywhere in the source. Error handling is absent, no recovery guidance, no categorization of retryable vs fatal errors. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels (WRITE, DESTRUCTIVE, READ_ONLY). Security concerns: tools like add_mcp accept env vars and headers as parameters, potential secret injection vector. No pagination support visible in list-like tools (library_search, library_find). Composition is reasonable, tools chain together (e.g., library_search → equip), but missing IDs in responses would force extra lookups.
Add an MCP server integration to the system. Can specify command, args, URL, environment variables, and headers.
A login wall on the shared computer, moved into the conversation: a live QR to scan, or a card to fill.
Change a bot's avatar: 'regen' | 'reset' | a look description | an image file the bot produced.
Start an asynchronous self-build; resolves as soon as the job is registered. Allows building a bot with a specific aspect (channel, skill, etc.), trigger, and change.
The round trip that proves an IM is really connected: hand out a code, then say whether it came back.
Every user-configurable thing in the product, atomized: get/set/add/remove. Targets: bot, matter (group), or profile. Fields for bot: autonomy(tell|prepare|do), notify, pinned, avatar. Fields for matter: title, summary, members, lead, notify, pinned. Profile is read-only.
No output schemas documented. Tools return data but LLMs cannot plan downstream calls or extract fields without guessing structure. Violates pattern:tool and pattern:response-shaper.
Vague parameter descriptions lacking format, range, or validation rules. E.g., 'configure' field param lists options inline ('autonomy(tell|prepare|do)') instead of enum constraints. 'harvest' selector param has no guidance on CSS selector syntax. Violates pattern:constrained-input.
Security risk: add_mcp accepts 'env' and 'headers' as parameters. Credentials passed as tool params will be logged and exposed in traces. Violates pattern:secret-injection.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
Hand the user an authorization card for a built-in connector (or report it's already connected / unavailable).
Put the credentials card for one IM into the bot's thread: the bot becomes its own bot over there. Connects a bot to an instant messaging channel (Slack, Feishu, etc.).
Create a new long-lived bot for the user — one more dedicated colleague on the team. Give one sentence in the user's own voice saying what it looks after, and the system generates its name, remit, skills and avatar; seconds later it can take work. You can hand it a first task at the same time.
Open a group for something that takes several people, with the user in it. Name the members (by name or id), optionally a lead (you by default), and either drop the task in at the same time or just open the group. Everything said there is visible to every member, but only the bot that is @-mentioned wakes up to answer; the lead pushes it along and pulls it together. The result stays on that thread, where you and the user can both see it.
Take the bot off an IM: stop its account there and forget the credentials.
Equip this bot with one pool entry, whatever kind it is: mount a manual, connect an MCP server, send a one-click card, download an asset pack. Installing dependencies is part of it.
Give this bot an existing integration (and put its tools in front of it right away).
Read a credential off a page in the shared browser straight into the bot's IM account / a connection's env (never through the model); or the setup facts for a target.
The same pool, searched by meaning as well as by words (library.ts). One embedding call, so: not per turn.
Lexical search over the curated skill library; empty query lists everything.
Steward only: this machine, and the machine the user is moving the bots to (if one was installed).
Remove an integration from the system.
No error handling guidance. Tools declare risk levels (WRITE, DESTRUCTIVE) but provide no recovery hints, retryability classification, or actionable error messages. Violates pattern:recovery-guide and pattern:error-classification.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Risk levels are declared in metadata but not in schema. LLMs cannot infer safety properties from tool definitions alone.