Loading ToolBench
Three tools with basic descriptions and partial schemas. Tool names follow verb_noun convention (start_kernel, connect_to_kernel, execute_code), which is good. However, descriptions are minimal (50-80 chars), parameter descriptions lack detail on constraints and formats, and output schemas are entirely undocumented. No error handling guidance, no parameter validation rules stated, and no indication of idempotency or side effects. The code shows functional implementations but the tool definitions lack the rigor needed for reliable LLM invocation.
Connect to an existing IPython kernel using its connection file.
Execute Python code on the connected IPython kernel.
Start a new IPython kernel using a connection file and automatically connect to it.
Output schemas completely undocumented. All three tools return strings with no structured schema definition. LLMs cannot plan downstream calls or extract typed data.
Parameter descriptions lack format/constraint details. 'connection_file' and 'code' parameters have minimal guidance on expected format, length, or validation rules. No mention of path traversal risks or code injection concerns.
No error handling guidance in tool descriptions. Errors are returned as plain strings (e.g. '❌ Connection file not found'). LLMs receive no actionable recovery steps or error classification (retryable vs fatal).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
execute_code tool lacks idempotency and side-effect documentation. Executing arbitrary Python code is inherently stateful and non-idempotent, but the description does not warn LLMs of this risk or suggest confirmation patterns.
No security guidance on code execution. execute_code accepts arbitrary Python strings with no mention of sandboxing, permission checks, or input sanitization. LLMs could be tricked into executing malicious code.