Loading ToolBench
Read-only Windows remote filesystem access via WinRM with MCP protocol support
Server has 6 tools with complete input schemas and descriptions. Naming follows verb_noun convention (connect, disconnect, list_*, find_*, read_*). Descriptions are detailed (100-250 chars) and explain WHEN to use each tool. However, output schemas are not documented, LLMs cannot predict response structure for downstream chaining. Error handling is minimal; no recovery guidance or actionable error messages. Parameters lack enums where appropriate (e.g., encoding should be constrained). No tool annotations (readOnlyHint, idempotentHint) despite all tools being read-only. Session management via session_id is sound, but no batch operations or pagination for find_files despite max 100 results.
Open a WinRM session to a Windows host and return a session_id; this is the required first step before any other tool. The AD password is elicited once and cached only in server memory, and the response includes computer name, OS version, and last boot time for triage.
Close an active WinRM session and release it; sessions are not auto-cleaned, so always disconnect when done. If you don't have the session_id, call list_sessions first.
Recursively find files matching a wildcard pattern, returning FullName, Size, and LastWriteTime as tabular text (max 100 results). Matches files only, not directories — use list_directory to browse a single folder.
List files and directories at a path as tabular text (Mode, LastWriteTime, Length, Name; max 200 entries, most recently modified first). Use this to explore one folder; use find_files to search recursively by pattern instead.
List active WinRM sessions with host, connection time, last used time, and command count. Use it to find a session_id or check whether you are already connected to a host.
Output schemas not documented. LLMs cannot predict response structure (fields, types, nesting) for downstream tool chaining or data extraction. E.g., connect() returns dict with 'computer_name', 'os_version', 'last_boot_time', not visible in schema.
No tool annotations despite all tools being read-only. Missing readOnlyHint on all 6 tools. Agents cannot distinguish safe tools from destructive ones without explicit hints.
Encoding parameter (read_file) accepts free-form string instead of enum. Valid values are hardcoded in VALID_ENCODINGS set but not exposed as schema constraint. LLMs may pass invalid encodings like 'latin1' or 'cp1252'.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | <=2025-11-25 | v2 |
Read file contents as numbered lines (max 500 per call) in ' 1|line' format. Range mode reads start_line through end_line; tail mode (tail=True) reads the last N lines and stays fast even on huge files.
Error handling lacks recovery guidance. No tool returns actionable error messages like 'Session not found. Call list_sessions() to find active sessions.' Raw errors force LLM to guess next steps.
find_files returns max 100 results but no pagination support. No limit parameter, no next_cursor, no total_count. Large result sets cannot be paginated; LLM cannot fetch additional results.