Loading ToolBench
Multi-agent verification MCP server for LM Studio. Audits answers with critic models and NLI claim-checking. Exposes tools for answer verification, second opinion consultation, and confidence scoring.
Verity presents two sophisticated verification tools with excellent, detailed descriptions that are LLM-optimized and actionable. Both tools have comprehensive input schemas with typed parameters. However, output schemas are not documented in the source code, and tool annotations (readOnlyHint, destructiveHint) are absent. The descriptions are exceptionally long and detailed (900+ chars each), which is at the upper bound but justified by complex operational workflows. Parameter descriptions are clear and specific. The tools follow the verb_noun naming pattern and have minimal ambiguity.
Consult two independent cross-family models (one per GPU) in parallel for a second opinion on the user's question, then run an analysis pass on NVIDIA comparing the two answers. Returns both answers plus a structured {agreements, disputes, table_html, table_md} analysis object. In auto mode the analysis also synthesises a final_answer. SOURCING NOTE: If the user also typed /verify in this turn, the answer you compose AFTER this tool returns must follow the verify_answer sourcing contract — every non-trivial fact backed by a fetched/working URL, structured [N],[author],[publisher],[year],[page],[url] citations, no fabricated fields. See verify_answer's description for the full contract. /second's parallel opinion does not exempt you from sourcing. INVOKE THIS TOOL when: - the user's latest message matches '/second' or 'second opinion' or 'ask the other model', - or you are about to commit to a non-trivial answer and want a parallel sanity check — call this tool once, near the start of your reasoning, BEFORE you write your final answer. COMPLEMENTARY WITH /verify: consult_second_opinion and verify_answer are independent tools, not alternatives. /second runs BEFORE you write your answer (parallel opinion); /verify runs AFTER (audit of what you wrote). If the user appended '/verify' to their question, you MUST still call verify_answer after composing your answer — even if you already called this /second tool proactively at the start. Calling /second does NOT replace or satisfy a /verify request. Skipping /verify when the user typed /verify is a contract violation.
Audits an answer with critic models + NLI claim-check. Returns a ready-to-paste Markdown block (answer echoed, verdict table, findings, follow-up prompt). WHEN TO CALL: user typed `/verify`, `/verifydeep`, or `/verifydeeper` in their latest message. Map to mode=standard / deep / deeper. ALSO CALL when your last assistant turn was a Verity block ending with the 'Awaiting your reply' prompt and the user replied affirmatively (yes / OK / `/verifydeeper` / sure) — call with mode='deeper'. If the user replied 'redraft', rewrite the answer (fetch-verifying every URL — do not fabricate sources) then call this tool again on the rewrite. If they replied 'no', do nothing. FLOW for `/verify` on a fresh question: 1. Write a substantive prose answer. Back non-trivial claims with fetched URLs cited as [N], [author], [publisher], [year], [page], [url]. Drop claims you can't source. 2. Call verify_answer with question + your prose answer. 3. Paste the returned Markdown block verbatim into chat. LM Studio collapses tool results by default; the user only sees what's in your assistant message body. 4. Stop. The block has its own follow-up prompt; wait for the user's reply, do not redraft unprompted. DO NOT: skip the tool call when /verify was typed; paraphrase the block (paste verbatim); redraft without explicit user consent; invent the verdict table from your own reasoning.
Output schemas not documented in source code. verify_answer and consult_second_opinion both return structured Markdown blocks and analysis objects, but the response structure is inferred from descriptions rather than declared in a schema field. LLMs cannot reliably parse undocumented output shapes.
No tool annotations present. Neither tool declares readOnlyHint, destructiveHint, or idempotentHint. verify_answer and consult_second_opinion are read-only (they audit/analyze, not modify state), and this should be explicitly signaled via annotations for protocol compliance and agent reasoning.
Parameter descriptions are exceptionally detailed (900+ chars total per tool), bordering on procedural documentation rather than parameter-level clarity. While the content is valuable, the format violates the pattern guideline of 10 - 1024 chars per description. This risks token waste and dilutes per-parameter signal. The detailed procedural instructions ('WHEN TO CALL', 'FLOW') belong in a separate usage guide, not embedded in parameter docs.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 78 | <=2025-11-25 | v2 |
No documented error handling or recovery guidance. The tools reference complex verification pipelines (critic models, NLI, consistency checks, perplexity re-sampling), but the source does not show what errors might occur (upstream timeout, malformed input, invalid mode) or how the LLM should recover. Error responses should include actionable guidance.
Parameter 'prior_context' in verify_answer is optional but has no default value and no guidance on size limits. The description says 'Keep under 24k tokens' but no max_length or pattern constraint is visible in the schema. Unbounded optional strings invite excessive context inclusion.