A Java/Spring Boot MCP server exposing TypeSafe Jev as typed-judgment tools
Five tools with clear, domain-specific purposes (classification, scoring, yes/no checks, patch gating, health). All have descriptions (50-150 chars, within baseline 34-392 range) and input schemas with type info. However, output schemas are undocumented, LLMs cannot predict return structure. Parameter descriptions are present but minimal (10-30 chars vs baseline 72 chars). No error handling guidance; JevToolException surfaces only a message string. Tool names are verb-noun style (jev_classify, jev_score) but lack action clarity, 'jev_' prefix is domain-specific but doesn't convey intent to unfamiliar LLMs. Missing: enum constraints, format specs, pagination, and recovery hints.
Ask a reusable yes/no question over state. Returns the probability of yes and a band: 'flag' at or above 0.70, 'clear' below 0.30, 'uncertain' between. Not for text generation.
Classify state into exactly one of the given labels, with a probability per label. Not for text generation.
Gate a patch before merge: verify each completion claim against the diff and the evidence supplied (e.g. test output), and score the diff's merge risk. Returns a verdict per claim plus an overall pass/review recommendation. Not for text generation.
Check connectivity to Jev and report the resolved model and round-trip latency.
Rate state against an ordered rubric, lowest to highest. The score is a probability-weighted zero-based index and can fall between levels. Not for text generation.
Output schemas undocumented. ClassifyResult, ScoreResult, CheckResult, GateResult classes exist in code but are not exposed in tool definitions. LLMs cannot predict return structure or plan downstream calls.
Parameter descriptions are minimal (10-30 chars). 'The text or state to classify' and 'Map of label name to description' lack format specs, constraints, or usage hints. Baseline is 72 chars; these are 30-50% of that.
No error recovery guidance. JevToolException wraps exceptions as plain strings. LLMs receive 'Connection failed' or similar with no hint: retry? Call health check first? Ask user? Violates recovery-guide pattern.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Tool names lack action clarity. 'jev_classify' and 'jev_score' are domain-specific but do not convey intent to unfamiliar LLMs. Consider 'classify_text_with_jev' or 'evaluate_text_against_rubric' for better discoverability.
No enum constraints or format specs. 'labels' parameter accepts any Map<String, String>; 'levels' accepts any List<String>. LLMs may pass invalid structures. Add JSON Schema constraints (minItems, pattern, etc.) and describe expected format in descriptions.