Loading ToolBench
MCP server giving Claude a stateful IPython kernel per session, with per-env isolation.
Server has 5 tools with clear verb-based names (execute, list_sessions, reset, shutdown, interrupt) and reasonable descriptions (avg 120 chars). All tools have input schemas with type definitions and parameter descriptions. However, output schemas are not documented, responses are inferred from code (ExecuteResult dataclass, dict returns). Parameter descriptions are minimal (10-30 chars) and lack constraints, ranges, or format guidance. Error handling exists but lacks recovery guidance. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk profiles (WRITE, IRREVERSIBLE, REVERSIBLE).
Run ``code`` inside the kernel for ``session``. Creates the session on first use. ``env`` selects the Python environment for new sessions: a bare name like ``"news-monitor"`` resolves to ``~/envs/<name>/bin/python``; anything containing ``/`` is treated as a path. On subsequent calls the session's existing env is used and a ``warning`` is returned if a different env was requested. Returns the result, stdout, or stderr string on success. Raises an error with the traceback on failure or timeout.
Send an interrupt to a running cell in ``session``.
List active sessions with their env, status, uptime, and execution count.
Restart the kernel for ``session`` (wipes state, keeps name and env).
Kill the kernel for ``session`` and forget the session.
Output schemas not documented. ExecuteResult and dict responses are inferred from code; LLMs cannot see what fields to expect. list_sessions returns list[dict[str, Any]] with no field documentation.
Parameter descriptions are minimal (10-30 chars) and lack constraints. 'session' param has no guidance on format or naming convention. 'env' lacks examples of valid formats (bare name vs path). 'timeout_s' has no min/max bounds.
Tool annotations missing. execute is WRITE, reset/interrupt are REVERSIBLE, shutdown is IRREVERSIBLE, but no readOnlyHint, destructiveHint, or idempotentHint annotations in schema. LLMs cannot infer safety profile.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2025-06-18+ | v2 |
Error handling lacks recovery guidance. reset/shutdown/interrupt return {'ok': False, 'error': str(exc)} on KeyError but do not suggest next steps (e.g., 'Call list_sessions() to see available sessions').
execute tool description mentions env resolution rules but does not document timeout behavior, partial output on timeout, or what 'result, stdout, or stderr string' means. Ambiguous return semantics.