Loading ToolBench
Server has 8 tools with complete input schemas and descriptions, but quality is uneven. Tool names follow verb_noun convention (connect, disconnect, read_screen, send_keys, send_key, set_cursor, upload_file, download_file). Descriptions are present but vary in clarity, some are verbose with implementation details (read_screen, send_key) while others are terse (disconnect). Parameter descriptions are generally adequate but lack constraints (e.g., send_key accepts 'Enter, F1-F24, PageUp, PageDown, Tab, Backtab, Clear, Help, Print, Attn' as free-form string, not enum). Error handling returns structured dicts with 'error' or 'status' keys, but lacks recovery guidance. No output schemas documented. Missing idempotency guarantees and confirmation patterns for destructive ops (upload_file, disconnect). Baseline: avg tool description 194 chars; this server averages ~180 chars. 100% of tools have descriptions (good), but only ~50% have fully constrained parameters.
Connect to an IBM i system via TN5250. Args: host: IBM i hostname or IP. Defaults to IBMI_HOST env var. port: TN5250 port. Defaults to IBMI_PORT env var or 23. use_ssl: Enable TLS. Defaults to IBMI_SSL env var. Returns the initial screen after connection.
Disconnect the active TN5250 session.
Download a file from the IBM i to the local system. Args: remote_path: Path to the file on the IBM i (IFS path). local_path: Destination path on the local system. Returns status with file details, or an error.
Read the current 5250 screen content. Returns structured data with: - screen: list of text rows (the visible display) - cursor: current cursor position {row, col} (1-based) - fields: list of input fields with position, length, value, and type - dimensions: screen size {rows, cols}
Send an attention/function key and wait for the host response. Args: key: Key name. Valid values: Enter, F1-F24, PageUp, PageDown, Tab, Backtab, Clear, Help, Print, Attn. Returns the updated screen after the host processes the key.
send_key parameter 'key' accepts free-form string instead of enum. Description lists valid values ('Enter, F1-F24, PageUp...') but no JSON Schema enum constraint. LLMs will hallucinate invalid keys like 'F99' or 'Escape'.
disconnect() has only 8-word description ('Disconnect the active TN5250 session.'). Under 20 chars of actionable detail. Does not explain when to call it, what happens to buffered state, or whether it's safe to reconnect immediately.
No output schemas documented for any tool. LLMs cannot predict response structure (e.g., read_screen returns {screen, cursor, fields, dimensions} but this is only in docstring, not in tool metadata). Agents must infer structure from examples.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
Type text into the current input field at the cursor position. Args: text: The text to type. Returns the updated screen state.
Position the cursor at the specified location. Args: row: Row number (1-based, top row is 1). col: Column number (1-based, leftmost column is 1). Returns the updated screen state.
Upload a file from the local system to the IBM i. Args: local_path: Path to the file on the local system. remote_path: Destination path on the IBM i (IFS path). Returns status with file details, or an error.
Error responses lack recovery guidance. Example: 'Not connected. Call connect() first.' is good, but 'Connection failed: {e}' (from connect tool) exposes raw exception text. Should categorize as retryable/user-fixable and suggest next steps.
Destructive operations (upload_file, disconnect) lack confirmation or dry-run support. Agents can accidentally overwrite remote files or drop sessions without explicit user approval. No idempotency guarantees documented.