Loading ToolBench
MCP server exposing an AI-slop evaluation gate with RAG grounding checks, rubric resources, and review prompts over stdio.
Two tools with complete input schemas and reasonable descriptions. score_slop has a clear, action-oriented description (71 chars); grade_rag is similarly well-described (82 chars). Both schemas properly typed with required fields. However, parameter descriptions are minimal (e.g., 'Draft / prose to score' lacks guidance on length, format, or expected content). Output schemas are not documented, LLMs cannot predict the structure of returned SLOP INDEX, verdict rows, or grading results. No error handling guidance. Tool names follow verb_noun pattern (score_*, grade_*) but are domain-specific and could benefit from clearer intent signaling. Composition is sound: two independent, single-purpose tools. Missing: output schema documentation, parameter constraints (e.g., max text length), error recovery guidance.
Deterministic RAG grounding/faithfulness score: fraction of answer tokens supported by context.
Score text for AI-slop tells (SLOP INDEX, verdict, per-rule rows, flagged spans).
Output schemas not documented. LLMs cannot predict structure of SLOP INDEX, verdict, per-rule rows, or grading results. Agents must guess at response fields, risking extraction errors.
Parameter descriptions lack actionable constraints. 'Draft / prose to score' does not specify max length, expected format, or content type. 'Use Claude live faithfulness check' does not explain when to use it or what ANTHROPIC_API_KEY requirement means.
No error handling guidance. If score_slop fails on malformed input or grade_rag cannot ground answer in context, LLM receives no recovery hint. Missing: 'If grading fails, try shorter context' or 'If score_slop times out, reduce text length.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
grade_rag 'live' parameter is optional boolean with no enum or default documented. LLM may pass invalid values. Should clarify: 'Default false (uses stdlib grader, no API key needed). Set true only if ANTHROPIC_API_KEY is set.'
Tool descriptions do not explain when to use each tool or how they compose. A user might call both score_slop and grade_rag on the same text without understanding their distinct purposes (slop detection vs. RAG faithfulness).