Loading ToolBench
Haskell-based infrastructure server providing PTY (pseudo-terminal) management and command execution capabilities through a domain model architecture
This MCP server has critical definition quality gaps. Tools are defined with minimal descriptions and incomplete schemas. The codebase shows tool parameter types (StringToolParams, PtyConnectToolParams, etc.) in Type.hs, but critical issues prevent higher scoring: (1) Tool descriptions are present but extremely brief (10-30 chars), far below the 50-200 char baseline for LLM optimization. (2) Input schemas lack formal JSON Schema structure, parameters are defined as Haskell records with Aeson deriving, but the actual schema objects exposed to MCP clients are not visible in the provided source. (3) Output schemas are completely undocumented, no indication of what these tools return or how results are structured. (4) No error handling guidance; no documentation of what each tool does beyond a one-liner. (5) Tool naming is inconsistent: 'pty-connect', 'pty-bash', 'pty-ssh', etc. use hyphens and generic prefixes (pty-) rather than action verbs (open_pty, start_bash_repl, etc.). The 'echo' tool is generic; 'pty-terminate' lacks clarity (terminate what? the current session?). (6) Security concerns are unaddressed: no indication of permission gating, audit logging, or input sanitization for these shell-execution tools.
Echo command that outputs a string value and invokes a callback
Connect to a pseudo-terminal running bash shell with optional arguments
Connect to a pseudo-terminal running Cabal REPL with project directory and optional arguments
Connect to a pseudo-terminal with a specified command and optional arguments
Connect to a pseudo-terminal running GHCi with project directory, optional startup file, and optional arguments
Send a message to the active pseudo-terminal
Connect to a pseudo-terminal via SSH with optional arguments
Connect to a pseudo-terminal running Stack REPL with project directory and optional arguments
Output schemas completely undocumented across all 10 tools. No indication of return types, field names, or structure. LLMs cannot plan downstream tool calls without knowing what data is available.
Input schemas not formally visible in MCP-compliant JSON Schema format. Parameter types are defined as Haskell records in Type.hs (StringToolParams, PtyConnectToolParams, etc.) but the actual JSON Schema objects exposed to clients are not shown. Inferred from Aeson deriving, but not verifiable.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 25 | <=2025-11-25 | v2 |
Connect to a pseudo-terminal via Telnet with optional arguments
Terminate the current pseudo-terminal connection
All tool descriptions are 10-30 characters, well below the 50-200 char baseline for LLM optimization. Descriptions lack WHAT the tool does, WHEN to use it, and prerequisites. Example: 'Terminate the current pseudo-terminal connection' (19 chars) does not explain how 'current' is determined or what state is preserved.
Tool naming is non-standard: uses hyphens (pty-connect) instead of underscores (pty_connect), lacks action verbs (pty-bash should be 'start_bash_repl' or 'open_repl_bash'), and uses generic prefixes. Naming should follow verb_noun convention (e.g., start_pty, send_input, close_pty).
No error handling guidance. No indication of what errors these tools can produce, whether they are retryable, or how to recover. Example: if pty-bash fails to start bash, the LLM receives no guidance on next steps.
Security concerns unaddressed: tools execute arbitrary commands (pty-bash, pty-ssh) with no documented permission checks, input sanitization, or audit logging. No indication of whether credentials are injected server-side or exposed as parameters.
Parameter descriptions are minimal or absent. Example: pty-connect 'arguments' parameter is described as 'Optional array of command arguments' but does not explain what happens if arguments are malformed, if shell metacharacters are escaped, or if the command fails.
Stateful PTY session management is opaque. Tools reference 'current' or 'active' pseudo-terminal (pty-terminate, pty-message), but there is no documentation of how sessions are tracked, multiplexed, or cleaned up. An LLM cannot reliably use these tools without understanding session lifecycle.