Loading ToolBench
AI Hallucination Detector — Formally verified trust scoring for LLM outputs. Analyzes AI-generated text for hallucination risk using linguistic signals, claim decomposition, and formally verified trust scoring.
TruthLens exposes 4 read-only analysis tools with clear, domain-specific descriptions (avg 180 chars). All tools have input schemas with type definitions and parameter descriptions. However, output schemas are not documented, tools return JSON strings without specifying field structure. Parameter descriptions are adequate but lack format constraints (e.g., minimum text length, response array bounds). No error handling guidance is visible (e.g., what happens if text is empty or responses list has <2 items). Tool naming is clear and verb-forward (analyze_, consistency_, extract_), but composition could be tighter, extract_claims_tool and extract_entities_tool are separate when they could be unified into a single extract_features tool. No tool annotations (readOnlyHint, idempotentHint) are declared despite all tools being read-only and idempotent.
Analyze AI-generated text for hallucination risk. Decomposes text into atomic claims and scores each for hallucination risk using linguistic signals (confidence, hedging, specificity, verifiability). Returns a trust score in [0, 1] with risk level and per-claim breakdown. No API keys or LLM calls required. Fast, local, formally verified.
Check consistency across multiple AI responses to the same prompt. Compares N responses and detects contradictions between them. Useful for identifying hallucinations by checking if multiple LLM outputs agree.
Extract atomic claims from text. Decomposes text into individual, independently verifiable claims. Useful as a preprocessing step before fact-checking or trust scoring.
Extract named entities (people, places, dates, numbers) from text. Identifies fact-checkable entities that can be cross-referenced against knowledge bases for verification.
Output schemas not documented. Tools return JSON strings but do not specify field structure, types, or required fields. LLMs cannot plan downstream operations or extract specific data without knowing response shape.
No error handling guidance. consistency_check returns a JSON error object for <2 responses, but other tools lack documented error cases (empty text, malformed input, service failures). LLMs cannot self-correct without actionable error messages.
Tool naming inconsistency: extract_claims_tool and extract_entities_tool use '_tool' suffix, breaking verb_noun convention. Should be extract_claims and extract_entities. Suffix signals uncertainty about tool identity.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
No tool annotations. All tools are read-only and idempotent, but lack readOnlyHint and idempotentHint declarations. Agents cannot infer safety properties without explicit annotations.
Parameter constraints missing. consistency_check.responses lacks minItems/maxItems; text parameters lack minLength. Unbounded inputs invite invalid calls (empty text, single response).