Loading ToolBench
MCP server for managing and discovering AI agents, MCP servers, skills, and models in a Kubernetes-based registry with deployment capabilities
The server exposes 17 tools with mostly complete schemas and descriptions. Naming is action-verb-first and generally clear (list_*, get_*, create_*, delete_*, etc.). Descriptions are present and substantive (avg ~150 chars), exceeding the 20-char minimum. However, there are consistent gaps: (1) Output schemas are NOT documented, the rubric requires structured output documentation, but this server provides no visibility into what fields tools return. (2) Parameter validation rules are underspecified, constraints like limits, enums, and format patterns are missing for several tools. (3) No per-parameter validation guidance for LLM self-correction. (4) Error handling is implicit; no recovery patterns or actionable error messages specified. (5) Several tools accept loosely-typed objects (e.g. 'config' as generic object) without field documentation. The tools are READ_ONLY safe, but the WRITE/DESTRUCTIVE ones (deploy_catalog_item, delete_deployment, delete_catalog) lack confirmation or dry-run patterns. Overall: solid naming and descriptions, but missing output schemas and error guidance push the score to lower B/upper C range.
Analyze an agent's full dependency tree including MCP servers and models. Returns structured data showing what the agent needs and what is available in the registry.
Create a new catalog entry. For servers use type='servers', agents use type='agents', etc. Note: only basic fields are supported via MCP; use the HTTP API for full spec including packages, transports, and tool configs.
Delete a catalog entry and all its versions. This is irreversible. Use list_catalog to confirm the resource name before deleting.
Delete a RegistryDeployment and remove all Kubernetes resources it manages. Use list_deployments to find the deployment name.
Deploy a catalog item to Kubernetes by creating a RegistryDeployment. First use list_catalog/get_catalog to find the resource name and version. Use resourceType='mcp' for MCP servers and 'agent' for agents.
Output schemas are not documented. No tool description specifies what fields are returned, their types, or structure. LLMs cannot plan downstream tool calls or extract required data without documented return schemas.
Destructive operations (delete_deployment, delete_catalog) lack confirmation or dry-run patterns. Agents can irreversibly delete resources without a chance to review or confirm. No error recovery guidance provided.
Generic object parameters ('config' in deploy_catalog_item, update_deployment_config) lack field documentation. LLMs cannot know which keys are valid, required, or what values to pass without field-level descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 60 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 25 | - | v1 |
Return catalog and deployment data for one or more resources to support planning their deployment, including available servers, agents, and existing deployments.
Get full details for a specific catalog entry by name. Returns spec, status, deployment info, endpoint URLs, and package configurations. Use version='latest' or omit for the current version.
Get details for a specific deployment by name, including managed Kubernetes resources, status, config, and target environment.
Get the full topology map of all clusters, environments, and discovered resource counts. Use this to understand what is running across your infrastructure before deploying or querying catalog.
Get total counts of all resources in the registry (servers, agents, skills, models). Use this for a quick overview of registry contents.
List catalog entries by type. Use type='servers' for MCP servers, 'agents' for AI agents, 'skills' for reusable skills, 'models' for model configs. Supports search, version filtering, and pagination.
List active RegistryDeployments - catalog items that have been deployed to Kubernetes. Shows deployment status, namespace, environment, and resource type. Filter by resourceType='mcp' or 'agent'.
List remote environments configured for discovery and deployment. Each environment represents a Kubernetes cluster or namespace where resources can be discovered or deployed.
Recommend AI agents from the catalog for a specific use case (uses LLM sampling to analyze the catalog)
Recommend MCP servers from the catalog for a specific use case (uses LLM sampling to analyze the catalog)
Force an immediate re-scan of remote clusters to refresh the catalog with newly deployed or removed resources. Useful after deploying something outside the registry.
Merge new key-value pairs into an existing deployment's config. Only specified keys are updated; others are preserved. Use get_deployment first to see current config.
Numeric parameters (limit, version) lack min/max bounds or format constraints. The rubric requires constraints on numeric parameters to prevent LLMs from passing absurd values (e.g. limit=999999 causing timeouts).
No error handling guidance or recovery patterns documented. Tools do not specify what errors are retryable, which are user-fixable, or what action to take on failure. LLMs cannot self-recover from failures without actionable error messages.
String enum parameters (type, resourceType) are not formally declared as enums in the schema. 'type' can be 'servers', 'agents', 'skills', or 'models', but no enum constraint prevents LLMs from hallucinating invalid values.
Missing pagination metadata. Tools like list_catalog and list_deployments accept a 'limit' but do not document whether results include a total count, next_cursor, or hasMore flag. Large result sets could blow the context window without explicit pagination guidance.
recommend_servers and recommend_agents rely on LLM sampling/analysis but do not document the mechanism, latency, or error behavior. If the LLM provider is unavailable, these tools will fail silently without recovery guidance.