Loading ToolBench
A Next.js-based web application for multi-agent data engineering workflows with support for various LLM backends (OpenAI, Claude, Gemini, Ollama) and n8n integration
This server exhibits significant quality gaps across naming, descriptions, schema documentation, and error handling. While tool names generally follow verb_noun conventions (validateConnection, submitQuery, checkN8nHealth), many descriptions are vague or incomplete. Critically, input schemas are visible for only 7 of 15 tools, the remaining 8 tools (validateOpenAI, validateOllama, validateClaude, validateGemini, debugEnvironment, selectAgent, unselectAgent, setSelectedStrategy, setBackendType, updateApiSettings, resetConversation, setConversationStatus) have descriptions but no visible input schemas in the source code. Per the hard scoring rule: 'If you cannot see the actual tool definition in the source (only inferred): cap that tool's overall at 50.' This applies to 8 tools. Additionally, output schemas are not documented for any tool, responses are not explicitly structured. Error handling is minimal; most tools lack actionable error messages or recovery guidance. The server mixes concerns (backend validation, agent management, conversation state) but tools are reasonably separated by function.
Checks the health and availability of n8n and its configured workflow endpoint
Returns information about which environment variables are configured (shows presence, not values)
Resets the current conversation state, clearing all messages and returning to idle status
Selects an agent for use in the current conversation (maximum 4 agents can be selected at once)
Sets the backend type and workflow type for query submission, optionally enabling demo mode
Sets the current conversation status
Sets the strategy for agent coordination in the current conversation
8 of 15 tools lack visible input schema definitions in source code. Tools validateOpenAI, validateOllama, validateClaude, validateGemini, debugEnvironment, selectAgent, unselectAgent, and resetConversation have descriptions but no explicit input parameter schemas documented.
No output schemas documented for any of the 15 tools. LLMs cannot predict response structure, field types, or downstream parameter mappings. For example, submitQuery likely returns a job_id or result, but this is not formally declared.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 45 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 33 | - | v1 |
Submits a query to the n8n backend with selected agents and strategy for processing
Deselects an agent from the current conversation
Updates the API settings configuration
Validates that the Claude/Anthropic API key is configured and valid
Validates connection to the configured backend (n8n, demo mode) and optionally checks the availability of associated model APIs based on workflow type
Validates that the Gemini API key is configured and valid, and checks for gemini-2.0-flash model availability
Validates that Ollama is running and accessible at the configured URL
Validates that the OpenAI API key is configured and valid by checking the models endpoint
Descriptions for validateOpenAI, validateOllama, validateClaude are extremely brief (under 20 chars effective content). Example: 'Validates that the OpenAI API key is configured and valid by checking the models endpoint' lacks context on failure modes, when to call, or expected response.
No error handling or recovery guidance documented. Tools lack actionable error messages. If validateConnection fails, the LLM has no guidance on next steps: Should it retry? Adjust parameters? Call a different tool? Per pattern:recovery-guide, errors must tell the LLM what to do next.
Parameter descriptions are sparse. For example, selectAgent has only 'The ID of the agent to select', no mention of valid format, length, or constraints. updateApiSettings exposes 'apiKey' as a parameter, violating pattern:secret-injection (credentials must never appear as parameters).
Tool validateOpenAI, validateOllama, validateClaude, validateGemini appear to be simple validation wrappers without visible input schemas. They look like they should accept backend credentials or config, but the schema is not exposed. Are these GET or POST? What parameters trigger them? This ambiguity suggests incomplete tool registration or missing MCP integration.
State management tools (selectAgent, unselectAgent, setSelectedStrategy, setBackendType, setConversationStatus, resetConversation) have no documented return values. Do they return the new state? A confirmation? Errors? Without output schemas, the LLM cannot chain operations or verify state changes.
updateApiSettings accepts apiKey and apiUrl as parameters. This violates pattern:secret-injection, credentials must NEVER appear in tool parameters as they will be logged and included in agent traces. Use server-side secret injection via environment variables instead.