MCP Server for Knowledge Distillation Teacher Student Tracker Skill - implements temperature-scaled soft targets, student-teacher Kullback-Leibler (KL) divergence, and combined task-distillation loss for agent capability retention
Single tool 'compute_distillation' has a reasonable description (145 chars) but critical schema deficiencies. Input schema is present with three parameters, all with type declarations and descriptions, meeting baseline requirements. However, NO output schema is documented anywhere, the tool returns a dict with 5 computed fields (total_loss, distillation_loss, hard_label_loss, student_top_prob, teacher_soft_entropy) but LLMs cannot see this structure in advance. Tool naming lacks action verb prefix (should be 'compute_distillation_loss' or 'evaluate_distillation'). No error handling guidance, no validation of input ranges (logits could be any float), no recovery paths. The implementation is mathematically sound but the MCP interface is incomplete.
Evaluates knowledge distillation step by computing temperature-scaled soft targets, student-teacher Kullback-Leibler (KL) divergence, and combined task-distillation loss
Output schema completely undocumented. Tool returns 5 fields (total_loss, distillation_loss, hard_label_loss, student_top_prob, teacher_soft_entropy) but LLMs cannot see this structure. Violates pattern:tool requirement that output schemas be documented.
Tool name lacks action verb. 'compute_distillation' is vague; should be 'compute_distillation_loss' or 'evaluate_distillation_step' to clarify what is computed and returned.
No input validation or error handling. Parameters accept any float for logits with no range checks, no guidance on valid ranges, no error messages if computation fails (e.g., NaN from log of zero). Violates pattern:recovery-guide.
Parameter descriptions lack format/constraint details. 'Student model logits' and 'Teacher model logits' do not specify expected array length, numeric range, or whether they must be equal-length. LLMs cannot infer these constraints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 37 | 2026-07-28+ | v2 |
Tool description does not explain WHEN to use it or what the output means. A user/LLM reading 'Evaluates knowledge distillation step...' cannot determine if this is for model training, evaluation, or inference. Missing context for tool selection.