MCP Server for Kahneman-Tversky KTO Utility Loss Skill. Evaluates KTO loss on unpaired binary feedback using prospect theory with loss aversion modeling.
Single tool with minimal documentation. Tool name lacks action verb (should be 'evaluate_kto_loss' or similar). Input schema present with proper types but no parameter descriptions. No output schema documented. No error handling guidance. No security considerations. Composition is simple (one tool, one job) but severely underdocumented for LLM consumption.
Evaluate KTO loss on unpaired binary feedback (thumbs up / down)
Tool name does not start with action verb. 'evaluate_kto_sample' is passive; should be 'evaluate_kto_loss' or 'compute_kto_loss' to signal the action clearly to LLMs.
All three input parameters lack descriptions. LLMs cannot infer that 'policy_logprob' is log probability of the policy model vs reference model.
No output schema documented. Tool returns a dict with 5 fields (implicit_reward, utility, kto_loss, is_desirable, satisfaction_probability) but LLMs have no way to know this structure in advance.
No error handling or recovery guidance. If invalid logprobs are passed (e.g., non-numeric), the tool will crash with a generic exception. No actionable error messages for LLMs.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | 2026-07-28+ | v2 |
Tool description does not explain when to use it or what the output means. LLMs cannot determine if this is a utility function for training, evaluation, or inference without domain knowledge.