Loading ToolBench
A local MCP server to evaluate Pharo Smalltalk expressions
This server has 11 tools with basic structure but significant gaps in schema completeness, parameter descriptions, and error handling. Most tools have docstring descriptions but lack detailed parameter constraints. Input schemas are present but minimal, parameters have type and description but lack enums, ranges, or validation guidance. No output schemas are documented. Error handling is absent; tools return raw string output with no structure, classification, or recovery guidance. The tool names follow verb_noun conventions (good), but many descriptions lack actionable context for LLM selection. Per-tool average: 52 (range 40 - 65). This is typical for early-stage community servers.
Evaluate a Pharo Smalltalk expression using the simple -e option. Args: expression: The Smalltalk expression to evaluate Returns: The result of the evaluation
Evaluate a Pharo Smalltalk expression using NeoConsole. Args: expression: The Smalltalk expression to evaluate command: The NeoConsole command to use (default: eval) Returns: The result of the evaluation
Get the comment of a Pharo class. Args: class_name: The name of the class to get the comment for Returns: The class comment
Get the definition of a Pharo class. Args: class_name: The name of the class to get the definition for Returns: The class definition
Get the list of method selectors for a Pharo class. Args: class_name: The name of the class to get method selectors for Returns: The list of method selectors
Get the source code of a specific method in a Pharo class. Args: class_name: The name of the class selector: The method selector (message name) Returns: The method source code
No output schemas documented. All 11 tools return bare strings with no structured response format. LLMs cannot reason about what fields to extract or plan downstream calls.
Three tools have ZERO input schema: quit_neo_console (line 50 in server.py), get_neo_console_command_history (line 194), shutdown_repl_session (line 201). No parameters documented means the LLM cannot discover what inputs are needed.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 46 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 36 | - | v1 |
Get the command history from the current NeoConsole session. Returns: The command history as a string, showing numbered entries of previously executed commands
Get a system metric from Pharo using NeoConsole. Args: metric: The metric to retrieve (e.g., 'system.status', 'memory.free') Returns: The metric value
Install a Pharo package using Metacello. Args: baseline: The baseline name of the package (e.g., 'Historia') repository: The repository URL (e.g., 'github://mumez/Historia:main/src') Returns: The result of the package installation
Send quit command to NeoConsole to terminate the REPL session. Returns: The result of the quit command
Shutdown the persistent NeoConsole REPL session and server. Returns: The result of the shutdown operation
Parameter descriptions lack validation constraints. 'The Smalltalk expression to evaluate' tells the LLM nothing about syntax rules, length limits, or unsafe patterns. 'metric' parameter (get_pharo_metric) has no enum of valid metrics, examples 'system.status', 'memory.free' are presented as open suggestions, not constraints. LLMs will hallucinate invalid metric names.
No error handling or recovery guidance. Tools return raw subprocess output or exceptions with no categorization (retryable vs fatal) and no hints for recovery. E.g., if NeoConsole server fails to start, the agent gets a raw exception, not 'NeoConsole server unreachable. Ensure PHARO_DIR is set and the Pharo image exists.'
Destructive/irreversible tools lack safeguards. install_package and shutdown_repl_session modify state with no dry-run, confirmation step, or warning. No confirmation_request pattern. An LLM could accidentally install the wrong package or kill the session.
Tool composition fragmented. evaluate_smalltalk_with_neo_console and evaluate_simple_smalltalk are separate tools that do nearly the same thing. No clear guidance on when to use which. Also, none of the tools that query Pharo state (get_class_definition, get_method_source, etc.) are integrated, an agent searching for a method must chain multiple discovery calls manually.
Description brevity. Tool descriptions average ~70 characters (baseline 194 chars for A+ tools). E.g., 'Get the definition of a Pharo class.' is generic, does not explain when to use it vs get_class_comment or get_method_list, what format the definition has, or whether it includes methods.
No idempotency guarantees. Core functions (evaluate_pharo_neo_console, install_pharo_package) do not document whether repeated calls with same input are safe. If an agent retries due to a timeout, install_package could double-install a dependency.