Loading ToolBench
AI agent runtime with WebSocket protocol, artifact handling, and secret management
RuntimeUse exposes 6 tools with significant definition gaps. Tool names lack action verbs (openaiHandler, claudeHandler, ArtifactManager are nouns, not verb_noun patterns). Descriptions exist but are generic and lack WHEN/WHY guidance. Input schemas are partially visible but incomplete: openaiHandler and claudeHandler show complex nested objects (outputFormat, signal, env) that lack proper JSON Schema type definitions and parameter-level descriptions. CommandHandler, DownloadHandler, and ArtifactManager have minimal schema visibility. uploadFile schema is nearly absent. No error handling guidance, no output schemas documented, no pagination support visible. Tool composition violates single-responsibility: openaiHandler bundles system prompt, user prompt, model selection, output formatting, secrets redaction, and cancellation into one tool.
Manages artifact uploads, downloads, and file tracking with ignore patterns
Executes shell commands with support for working directory, environment variables, and output streaming
Downloads files from URLs and automatically unzips .zip files
Agent handler that runs Claude agents with code tools and structured output support
Agent handler that runs OpenAI agents with code interpreter, shell, and web search tools
Uploads a file to storage
Tool names are nouns, not action verbs. 'openaiHandler', 'claudeHandler', 'ArtifactManager' do not follow verb_noun convention (e.g., run_openai_agent, run_claude_agent, manage_artifacts). LLMs cannot infer intent from noun-only names.
Input schemas lack proper JSON Schema type definitions. openaiHandler and claudeHandler show nested objects (outputFormat, signal, env) with no 'type' field declarations. Parameters like 'signal' (AbortSignal) and 'env' (object) are not formally typed.
ArtifactManager has no visible input schema. Tool definition shows only description and file path, with no parameter documentation or type information.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., what does openaiHandler return? structured object? raw text? error details?).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 26 | <=2025-11-25 | v2 |
Tool descriptions are generic and lack WHEN/WHY guidance. 'Agent handler that runs OpenAI agents' does not explain when to use openaiHandler vs claudeHandler, what models are supported, or what structured output means.