MCP Server for Semantic Entropy Hallucination Estimator Skill. Estimates semantic entropy across multiple completions to detect hallucinations using bidirectional entailment semantic clustering.
Single tool with incomplete schema and minimal descriptions. Tool name lacks action verb (estimate_semantic_entropy is acceptable but borderline). Input schema present but parameter descriptions are extremely terse (under 20 chars each). No output schema documented. No error handling guidance. No parameter constraints (enums, ranges, formats). Tool does one thing but definition quality is well below production baseline.
Estimate semantic entropy across multiple completions to detect hallucinations
Parameter descriptions are trivial (under 20 chars). 'List of sampled string responses' and 'Optional list of log likelihoods for each response' provide minimal context for LLM parameter selection.
No output schema documented. Callers cannot predict the structure of returned fields (semantic_entropy, cluster_count, is_hallucination, etc.). LLMs must infer output shape from unstructured response.
No input validation or error handling guidance. Tool accepts empty responses list but raises ValueError, error message is not actionable for LLM recovery. No guidance on minimum/maximum response count or expected log_likelihoods format.
Tool description lacks WHEN to use it and prerequisites. Does not explain that responses should be semantically diverse samples, or that log_likelihoods must match response count. No guidance on entropy_threshold interpretation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
No parameter constraints. responses array has no minItems/maxItems; log_likelihoods has no format or range specification. LLMs can pass arbitrary array sizes or invalid numeric values.