Loading ToolBench
MCP server for Ray cluster monitoring and management
Rayatouille provides 11 read-only Ray cluster inspection tools with basic descriptions and minimal parameter documentation. All tools are properly named with action verbs (ray_*), but descriptions are generic and lack LLM-optimized guidance. Most tools have empty input schemas (emptySchema = {"type":"object"}), providing no parameter constraints. Tools 7-11 have partial schemas with required fields and descriptions, but lack type constraints and validation guidance. No error handling guidance, no output schema documentation, and no tool annotations. The codebase shows competent Go implementation but weak tool interface design for agent consumption.
Get stdout logs for a specific actor by actor ID
List cluster events with severity, timestamp, source, and message
Get Ray cluster health: version info, node counts, resource totals, job status summary
Get logs for a specific Ray job by submission ID
List all Ray actors with state, class name, job ID, PID, and node
List all Ray jobs with status, submission ID, entrypoint, and timing
List all Ray cluster nodes with state, IP, resources, and labels
Six tools (ray_cluster_health, ray_list_jobs, ray_list_nodes, ray_list_actors, ray_serve_status, ray_cluster_events) use empty input schema (type:object with no properties). This provides zero parameter constraints or guidance to LLMs.
Tool descriptions are generic and lack LLM-optimized guidance. Examples: 'Get Ray cluster health: version info, node counts, resource totals, job status summary' (79 chars) and 'List all Ray jobs with status, submission ID, entrypoint, and timing' (67 chars). Descriptions should explain WHEN to use the tool and what to do with results. Baseline: 194 chars average for A+ tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
Get the contents of a specific log file from a node
List available log files for a specific node
Get Ray Serve application status, deployments, and replica health
Get task summary for a specific Ray job by job ID
No output schema documentation. Tools return JSON (mustJSON helper) but LLMs cannot infer field names, types, or structure.
No error handling guidance. Tools return textResult(fmt.Sprintf("Error: %v", err)) with raw error messages. Try search_users() with a partial name.' Current errors give agents no recovery path.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All 11 tools are read-only but this is not declared in the tool definition. Per current MCP spec (2026-07-28), tool annotations enable agents to reason about safety and retry behavior.