Loading ToolBench
TypeScript Agentic Orchestration HTTP server and framework for building multi-agent workflows with MCP tools and vector stores
Agent Orcha provides 14 tools with mixed quality. Most tools have descriptions (good), but parameter schemas are incomplete or missing in several cases. Tool naming follows verb_noun conventions reasonably well (ask_user, canvas_write, integration_post). However, critical issues emerge: (1) Several tools lack visible input schema definitions in the source code provided (knowledge_entity_lookup_*, knowledge_graph_schema_*, knowledge_search_*, knowledge_sql_*, knowledge_traverse_* are template-based with [name] placeholders, making schema validation impossible). (2) Output schemas are not documented anywhere. (3) Error handling and recovery guidance are absent. (4) Parameters lack type constraints (enums where appropriate). (5) Some descriptions are too generic or missing actionable context. The server shows decent effort on naming and basic descriptions, but falls short on parameter rigor and schema completeness required for production use.
Wrapped agent tool allowing agents to be called by other agents in ReAct workflows.
Ask the user a question and wait for their response. Use when you need information that was not provided in the original request or when clarification is needed.
Append content to the existing canvas. Use this to incrementally build documents, add new sections, or extend code. The canvas must already be open (via canvas_write). The appended content uses the same format as the original canvas_write call.
Write content to the canvas side pane. Replaces any existing canvas content. Use this for documents, articles, reports, code, HTML pages/games, and any substantial output. For HTML apps/games, use format "html" so they render live. For source code (Python, JS, etc.), use format "code" with the appropriate language. For documents and articles, use format "markdown".
Send an email to a recipient. Use this to compose and send emails.
Generate an image using [name].
Output schemas completely undocumented. No tool documents what fields it returns, their types, or structure. LLMs cannot plan downstream tool calls or extract needed data without this information.
Template-based tool names (knowledge_entity_lookup_*, knowledge_graph_schema_*, knowledge_search_*, knowledge_sql_*, knowledge_traverse_*) use [name] placeholders in descriptions and lack explicit schema definitions. Cannot verify schema validity without seeing concrete registrations.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 62 | <=2025-11-25 | v2 |
| 2026-03-09 | C | 66 | - | v1 |
Get recent messages or emails from your integration. Use this to understand the current conversation context.
Post a message to your integration channel. Use this when asked to report, share, or send information to your channel.
Find entities in [name] graph by name, ID, or type. Use knowledge_graph_schema_[name] first to discover available types.
Get the schema of [name] graph: entity types, relationship types, properties, and counts. Call this first to understand available types before using lookup or traverse.
Semantic search over [name]. Use descriptive queries (10+ words) for best results.
Run a readonly SELECT query against [name] database. Only SELECT allowed.
Traverse relationships around an entity in [name] graph. Returns connected nodes within N hops. Use entity_lookup first to find IDs.
Save or update your long-term memory. Provide the COMPLETE memory content that should be persisted. This replaces the entire memory file. Use this to remember important facts, user preferences, and key context across conversations.
Missing parameter constraints (enums). Tools like canvas_write accept format='markdown|html|code' but no enum constraint prevents LLM from hallucinating invalid formats like 'json' or 'xml'.
No error handling guidance. Tools lack documentation on failure modes, error recovery, or actionable next steps. E.g., integration_context offers no guidance on what to do if no context exists.
Descriptions lack actionable context. generate_image description 'Generate an image using [name]' is incomplete, missing LLM usage guidance, prompt best practices, output format, cost, or latency expectations.
knowledge_sql_* accepts free-form queries with no injection safeguards documented. No mention of SQL escaping, parameterized queries, or how malicious SQL is prevented. Critical for security.
No permission declarations. Tools like email_send and integration_post lack scope documentation (e.g. 'requires: write:email', 'requires: write:channel'). Enables least-privilege misconfiguration.
No idempotency guarantees documented. Agents may retry email_send or integration_post on transient failures, risking duplicate sends. No idempotent flag or guidance provided.
Parameter descriptions lack format constraints. knowledge_sql_* 'query' param has no guidance on dialect (SQLite, MySQL, PostgreSQL?), max size, or timeout. knowledge_search_* 'query' recommends 10+ words but does not enforce it.