Loading ToolBench
Single tool 'run-code' has a basic schema with enum constraint on languageId, but critical gaps undermine quality. Tool description is minimal (47 chars, below 50-200 char baseline for LLM optimization). Parameter 'code' lacks format/length guidance. No output schema documented, LLMs cannot predict response structure. No error handling guidance (e.g., 'if execution fails, check syntax'). No mention of irreversible consequences despite RISK: IRREVERSIBLE designation. Missing idempotency/confirmation patterns for a destructive operation. Schema is present but sparse; descriptions are present but insufficient.
Run code snippet and return the result.
Tool description is too brief (47 chars). LLM cannot determine when to use this tool vs. alternatives, or understand consequences. Baseline: 50-200 chars for LLM optimization.
No output schema documented. LLMs cannot predict response structure or plan downstream operations. Baseline: 100% of A+ tools document return types.
Irreversible operation (code execution, file creation) lacks confirmation/dry-run pattern and error recovery guidance. No mention in description that this modifies system state.
Parameter 'code' has no format constraints, length limits, or validation guidance. LLMs may pass arbitrarily large or malformed code. No mention of supported syntax or error handling.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | 2026-07-28+ | v2 |
No error classification or recovery guidance. If code execution fails, LLM receives raw error with no actionable next step. Baseline: errors must guide LLM on retry vs. user-fixable vs. fatal.