Loading ToolBench
A Model Context Protocol server for Common Metadata Framework (CMF) API Server
The CMF MCP server presents a moderately functional tool suite with consistent structure but significant gaps in description quality and schema detail. All 8 tools follow a verb_noun naming convention (cmf_show_*, cmf_execution_*, cmf_artifact_*, cmf_show_model_card), which is positive for LLM intent inference. However, tool descriptions are terse (averaging ~80-100 chars, below the 194-char baseline for A+ tools), and parameter descriptions are minimal or missing in several cases. Input schemas are visible and use proper type annotations (Optional[List[str]], str), but lack depth regarding constraints, valid ranges, and format specifications. No output schemas are documented in the source, forcing LLMs to infer response structure. Error handling is absent, no recovery guidance, no categorization of failure modes, no retry logic documented. Security appears sound (no credentials in params, environment-based config), but tool annotations (readOnlyHint, idempotentHint) are not present, limiting LLM reasoning about safety.
Fetch artifact lineage tree for a specific `pipeline` on Common Metadata Framework (CMF) server. Must have run cmf_show_pipelines and validated that requested `pipeline` exists before running this.
Fetch execution lineage for a `selected_uuid` and `specific_pipeline`. Must have run cmf_show_pipelines and validated that `selected_pipeline` exists AND cmf_show_execution_detail to validate that `selected_uuid` before running this.
Retrieve a list of artifact types on Common Metadata Framework (CMF) server
Retrieve artifacts of a specific `artifact_type` for a given `pipeline` on Common Metadata Framework (CMF) server
Provides Executions details for a given pipeline in CMF server
Lists executions for a Pipeline in CMF Server
Missing output schema documentation. No tool explicitly documents what fields are returned or the structure of responses. LLMs cannot plan chained calls or extract IDs needed for downstream tools (e.g., execution UUID to pass to cmf_execution_lineage).
Minimal parameter descriptions. The cmfClient_instances parameter appears across all tools with identical wording ('Optional list of cmfClient_instances to query...'), but lacks context about what happens if None vs when to specify multiple. Parameter descriptions should be 50-100 chars and actionable.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 8 | - | v1 |
Retrieve the model card for a Model Artifact type (identified by its ID) from Common Metadata Framework (CMF) server
Lists all Pipelines in Common Metadata Framework (CMF) server
Tool descriptions are terse (40-70 chars). WHEN should the LLM call cmf_show_executions vs cmf_show_executions_list? Are they synonyms or serve different purposes? Baseline A+ tools average 194 chars with explicit WHAT/WHEN/prerequisites guidance.
No error handling or recovery guidance. No indication of what errors could occur (invalid pipeline name, auth failure, network timeout) or how the LLM should respond. E.g., if a pipeline_name is invalid, what does the tool return?
No tool annotations (readOnlyHint, idempotentHint, destructiveHint). All tools appear read-only, but this is not formally declared. LLMs cannot reason about idempotency or safety without annotations.
Parameter constraints and formats not specified. The selected_uuid param in cmf_execution_lineage says '(first 4 characters will be used)' in the description, this should be encoded as a length constraint or pattern in the schema, not a comment.
Redundant or confusingly similar tool names. cmf_show_executions and cmf_show_executions_list both describe fetching execution details, the naming distinction is unclear. Does one return a summary, the other full details? This ambiguity forces LLM guesswork.
Prerequisite dependencies not enforced. cmf_execution_lineage description states 'Must have run cmf_show_pipelines and validated...before running this.' These are informal comments, not tool logic. An LLM might skip validation and pass an invalid UUID, resulting in avoidable failures.