Loading ToolBench

Reading the current corpus…

verity MCP server · D · ToolBench

← Browse

verity

github.com/jsfaulkner86/verity

LLM confidence scoring layer — score, verify, and triage any LLM response

<=2025-11-25STDIO3 tools

52/100

Definition Quality67

Protocol Readiness47

Supportability31

Spec posture

Inferred effective spec <=2025-11-25·current baseline 2026-07-28

Static source inference · medium confidence · detected: Logging

Deprecated protocol patterns detected

Summary

Verity defines 3 tools with complete input schemas and descriptions. However, parameter descriptions are sparse or missing entirely. score_response has 6 parameters but only 2-3 have substantive descriptions; extract_claims and get_hitl_decision lack parameter-level detail. Output schemas are not documented. Error handling is minimal, no recovery guidance or actionable error messages. Tool names are clear (verb_noun pattern), but the server lacks composition guidance and does not document what fields downstream tools should expect.

Tools (3)

extract_claimsread onlysource verified67/100

Decompose an LLM response into atomic verifiable claims with metadata.

get_hitl_decisionread onlysource verified62/100

Given a ScorecardResult, return the HITL decision (ACCEPT / REFINE / REJECT / ESCALATE) and an optional refinement prompt.

score_responseread onlysource verified72/100

Score an LLM response for source grounding, factual consistency, claim specificity risk, and hedging calibration. Returns a full ScorecardResult with HITL recommendation.

Issues (5)

HIGHtool-description ↗

Parameter descriptions incomplete or missing. score_response has 6 parameters; only 'response_text' and 'prompt_text' have substantive descriptions. 'sources', 'source_model', 'domain', 'request_id' lack detail on format, constraints, or when to use them.

score_responseextract_claimsget_hitl_decision

HIGHresponse-shaper ↗

Output schemas not documented. Tools return ScorecardResult, claims array, and HITL decision objects, but the response structure is not declared in the tool definition. LLMs cannot plan downstream calls without knowing what fields to expect.

score_responseextract_claimsget_hitl_decision

HIGHrecovery-guide ↗

No error handling guidance. Tools do not document what errors can occur, whether they are retryable, or what the LLM should do next. A malformed scorecard in get_hitl_decision will fail with a Pydantic validation error, not a user-friendly message.

score_responseextract_claimsget_hitl_decision

Recommendations

Add substantive descriptions to all parameters. For 'sources', explain format (list of strings, objects with title/content, or URLs). For 'source_model', list valid identifiers (gpt-4o, claude-sonnet-4-6, etc.). For 'domain', explain scoring differences per domain.
Document output schemas for all three tools. Specify the structure of ScorecardResult (fields: overall_score, dimensions, claims, phi_flagged, domain), claims array (fields per claim), and HITL decision (fields: decision, refinement_prompt).
Add error handling guidance. E.g., 'If response_text is empty, returns error: "response_text required and must be non-empty". If domain is invalid, returns error with valid options.'
Clarify parameter constraints. For 'sources', specify max array length. For 'request_id', specify format (UUID, alphanumeric, etc.). For 'response_text' in extract_claims, specify max length and supported languages.
Add composition hints. E.g., 'score_response returns a ScorecardResult; pass it to get_hitl_decision to receive a HITL recommendation. Use extract_claims first to decompose the response into verifiable claims.'

Spec posture evidence

Inferred effective spec: <=2025-11-25.

Relies on Logging (deprecated) - log to stderr or use OpenTelemetry

Score history

Overall score trend

First recorded score · v2 rubric

52/100

Scored	Grade	Overall	Spec posture	Rubric
2026-09-23	D	52	<=2025-11-25	v2