Loading ToolBench
MCP server that exposes OllaBridge Control Plane tools for node enrollment, runtime management, and gateway health monitoring
Three tools are defined with basic schemas and descriptions, but quality is significantly below production standards. All three tools have descriptions (10-30 chars), which fail the baseline of 34-392 chars for A+ tools. Schemas are present but lack depth, parameters are minimally described (1-sentence, <20 chars each). No output schemas are documented. Tool names follow verb_noun convention (create, list, check) but descriptions are too terse to guide LLM selection. No error handling guidance, no input validation constraints, and no examples of chaining IDs across tools. The server exposes a hardcoded base_url default ('http://127.0.0.1:11435') across two tools, which is environment-dependent but not parametrized per-request, this violates the chat-data-model pattern by requiring agents to know internal deployment details.
Create a short-lived node enrollment token.
Check gateway health.
List currently connected runtimes.
Descriptions critically short (10 - 30 chars vs. baseline 34 - 392 chars). 'Create a short-lived node enrollment token' is 47 chars but lacks WHEN to use it, WHAT it returns, or dependency hints. Descriptions must answer: What does it do? When should the LLM call it instead of a similar tool? What does it return?
Parameter descriptions missing or generic. 'ttl_seconds' has description 'Token time-to-live in seconds' (24 chars) but does not specify valid range, default behavior on omission, or consequences of short/long TTLs. 'base_url' lacks any indication of format, requirement, or what happens if it is unreachable.
No output schemas documented. Each tool claims to return something (a token, a list of runtimes, health status) but the response structure is not visible in the code excerpt. LLMs cannot plan downstream tool calls or extract chaining IDs (e.g., which fields in the runtime list response identify a runtime?) without documented output schemas.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 37 | 2026-07-28+ | v2 |
Hardcoded base_url default ('http://127.0.0.1:11435') couples tools to a specific deployment. This violates the chat-data-model pattern by requiring agents to know internal deployment topology. Two tools (runtimes.list, gateway.health) both require this param with the same default, tight coupling suggests these should accept base_url from the MCP server context (e.g., via server metadata or environment injection), not as a tool parameter.
No error handling guidance. Tools do not describe failure modes or recovery paths. If a health check fails, does the LLM retry? If enrollment token creation fails due to rate limits, should it wait? Should it ask the user? No documented classification (retryable, user-fixable, fatal).
Parameter ttl_seconds lacks constraints. No min/max specified. Default 3600 (1 hour) is stated as a JSON default, but what if an LLM passes 1 (1 second) or 31536000 (1 year)? Are there API limits? The description should say 'ttl_seconds: integer, 60 - 86400 (1 minute to 1 day), default 3600'.
base_url parameter does not validate format or describe what the LLM should pass. Is it always http://host:port? Can it include a path? Can it be https? Can it be a DNS name or only an IP? The description 'OllaBridge gateway base URL' is too generic for safe LLM usage.
Tool names are domain-specific and lack clarity on how they relate. ollabridge.enroll.create creates a token for what? To enroll a what? The description says 'node enrollment token', but 'node' is never defined. What is a node in the OllaBridge context? Is it a worker? A client? A gateway? LLMs familiar with OllaBridge will infer correctly; a new user of the MCP server will not.