Loading ToolBench
MCP server for Red Hat OpenShift AI providing tools for cluster management, model training, inference deployment, workbench management, and data operations
RHOAI MCP Server demonstrates good definition quality with well-structured tool naming (verb-noun pattern), comprehensive descriptions, and properly typed schemas. All 14 tools are explicitly defined with input schemas and descriptions. However, several tools lack output schema documentation, some parameters could be better constrained with enums, and error handling guidance is minimal. The server follows many agentic tool patterns effectively but has gaps in composition optimization and output documentation that prevent a higher score.
Get a compact cluster overview optimized for context windows. Returns a minimal summary of the entire cluster including project count, workbench status, model deployment status, and resource availability. This is the most efficient way to get an overview before drilling down.
Create an S3 data connection. Creates a secret with the appropriate RHOAI labels and annotations that can be used by workbenches and pipelines to access S3 storage.
Delete a data connection. WARNING: Workbenches or pipelines using this connection will lose access to the associated data source.
Complete cluster exploration in one call. Returns comprehensive overview of the entire RHOAI cluster including all projects, their resources, and any issues detected. Use this as the first tool when exploring an unfamiliar cluster.
Get detailed information about a data connection. Sensitive values like secret keys are masked for security.
Output schemas not documented for any tool. Tools return responses but schema structure (fields, types, examples) is not formally specified. This forces LLMs to infer output fields, risking downstream tool call failures when expected IDs or references are absent.
Enum constraints declared in descriptions, not formalized in schema. resource_type, use_case, optimization_profile, verbosity all have fixed valid values documented in text but not as JSON Schema enums. This reduces machine readability and forces LLMs to parse description text to discover constraints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | B | 74 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Get a single example item by name.
List data connections in a Data Science Project with pagination. Data connections are secrets with RHOAI-specific labels that provide credentials for accessing external data sources like S3 buckets.
List example items (ConfigMaps labelled rhoai.io/example=true). This is a demonstration tool from the contributor blueprint.
List just resource names without full metadata. Returns only the names of resources, which is the most token-efficient way to get a list of available resources.
Get status for multiple resources in a single call. Efficiently retrieves status for multiple resources, reducing the number of tool calls needed.
Get a compact project status optimized for context windows. Returns a minimal summary of a specific project including workbench, model, and pipeline status without full resource details.
Get LLM model recommendations from Planner. Runs the full Planner recommendation flow: extracts intent from natural language, builds technical specifications, and returns four named recommendations: top_performance (lowest latency), top_cost (cheapest), top_balanced (weighted composite), and top_quality (highest quality score).
Get quick status check for any resource. Provides minimal status information for a specific resource without full details. Use this for quick health checks.
Get recommended tools and workflow for a given intent. Given a natural language description of what you want to do, returns the recommended tools and typical workflow steps.
Error handling guidance is minimal. Tools have no documented recovery paths. E.g., suggest_tools or recommend_model failures do not explain what to do next. Pattern: recovery-guide and error-classification missing.
AWS credentials (aws_access_key_id, aws_secret_access_key) passed as tool parameters in create_s3_data_connection. While parameters flow through the implementation, documentation should emphasize that in production, these should be injected server-side via environment or vault, not exposed as LLM-visible parameters. Risk: credentials may appear in agent traces.
No per-tool permission/scope declarations. Tools like delete_data_connection and create_s3_data_connection do not declare required capabilities (e.g., 'write:connections', 'admin:cluster'). This prevents least-privilege agent configuration and clear audit trail labeling.
Tool composition gaps: suggest_tools returns 'recommended tools and workflow steps' but output schema is not documented. If suggest_tools returns a list of tool names, does it also return the tool signatures or only names? This forces agents to rely on cached tool definitions and risks calling tools with wrong parameters.