MCP Server implementation for Conformal Prediction Coverage Guarantee Skill. Exposes tools to calibrate conformity scores and generate prediction sets.
Two tools with minimal descriptions (both under 100 chars) and incomplete parameter documentation. Input schemas present but lack parameter descriptions entirely, LLMs cannot infer what 'prob_distributions' or 'prob_distribution' objects should contain. No output schema documented. No error handling guidance. Tool names lack action verbs (calibrate_conformal, predict_set are domain-specific but not verb_noun pattern). No parameter constraints, enums, or validation hints. Baseline: 194 chars avg description, 72 chars avg param annotation, this server is 50% below baseline.
Calibrate non-conformity threshold from calibration set
Generate prediction set with guaranteed coverage
Parameter descriptions completely missing. 'true_labels', 'prob_distributions', 'alpha', 'prob_distribution' have no descriptions in schema. LLMs cannot infer what structure prob_distributions should be (array of dicts with what keys?) or how to construct prob_distribution objects.
Tool descriptions are too brief (both ~60 chars, baseline 194 chars). 'Calibrate non-conformity threshold from calibration set' lacks WHEN to use, WHAT it returns, and prerequisites. 'Generate prediction set with guaranteed coverage' omits that calibration must happen first.
No output schema documented. Callers cannot know what fields to expect from responses. calibrate_conformal returns {status, quantile, alpha} but this is not declared. predict_set returns {prediction_set, set_size, calibrated_threshold, significance_level, guaranteed_coverage} but undocumented.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 37 | 2026-07-28+ | v2 |
No error handling or recovery guidance. If calibrate_conformal is called with mismatched array lengths, it raises ValueError('Mismatched or empty calibration data') but the tool does not catch or format this for LLM recovery. predict_set raises RuntimeError if not calibrated first, no guidance on what to do.
Tool names lack action verbs. 'calibrate_conformal' and 'predict_set' are domain-specific but do not follow verb_noun pattern (e.g., 'calibrate_threshold', 'generate_prediction_set'). Baseline: 90% of A+ tools start with action verb.