Loading ToolBench
Server has 7 tools with basic verb-noun naming (kubectl_get, kubectl_logs, etc.). All tools have descriptions and input schemas are present with type definitions. However, descriptions are generic and lack LLM-optimized guidance on WHEN to use each tool vs. alternatives. Parameter descriptions exist but are minimal (avg ~40 chars vs. baseline 72). No output schemas documented. Error handling returns JSON but lacks recovery guidance. No tool annotations (readOnlyHint/destructiveHint). Schemas are present but sparse, many parameters lack constraints (enums, ranges). Overall: functional but below production baseline.
Apply a Kubernetes manifest. Args: manifest: YAML manifest content namespace: Namespace to apply to (optional) dry_run: Dry run mode ('none', 'client', 'server') Returns: Apply result
Get or set the kubectl context. Args: context: Context name to switch to (optional, lists contexts if not provided) Returns: JSON with context information
Create a Kubernetes resource. Args: resource: Resource type (deployment, service, configmap, secret, etc.) name: Resource name namespace: Namespace (optional) image: Container image (for deployments) dry_run: Dry run mode ('none', 'client', 'server') Returns: Create result
Get detailed information about a specific resource. Args: resource: Resource type (pod, deployment, service, node, etc.) name: Resource name namespace: Namespace (optional) Returns: Detailed resource description
Get Kubernetes resources. Args: resource: Resource type (pods, deployments, services, nodes, etc.) name: Specific resource name (optional) namespace: Namespace (optional, uses default if not specified) all_namespaces: Get resources from all namespaces output: Output format (wide, yaml, json, name) selector: Label selector (e.g., 'app=nginx') field_selector: Field selector (e.g., 'status.phase=Running') Returns: Resource information
No output schemas documented. LLMs cannot predict response structure or plan downstream tool calls. kubectl_get returns raw kubectl output (string), not structured JSON with typed fields.
Parameter descriptions lack actionable constraints. 'output' param accepts 'wide, yaml, json, name' but no enum constraint. 'dry_run' accepts 'none, client, server' but no enum. LLMs may hallucinate invalid values.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). kubectl_apply and kubectl_create are destructive but unmarked. Agents cannot distinguish safe reads from risky writes without parsing descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
Get logs from a pod. Args: pod: Pod name namespace: Namespace (optional) container: Container name (for multi-container pods) previous: Get logs from previous container instance tail: Number of lines to show (default 100) since: Only logs newer than duration (e.g., '5m', '1h') follow: Stream logs (limited to 10 seconds for safety) Returns: Pod logs
Check kubectl connection status and current context. Returns: JSON with connection status, current context, and cluster info
Error handling returns generic JSON errors without recovery guidance. 'error: <stderr>' tells LLM nothing about whether to retry, ask user, or abort. No error classification (retryable vs. fatal).
Parameter descriptions are minimal (avg ~40 chars vs. baseline 72). E.g., 'namespace: Namespace (optional)' lacks guidance on default behavior or when to specify. 'selector: Label selector (e.g., app=nginx)' includes example value that LLM may reuse literally.