Loading ToolBench
A fact-checking tool that verifies statements against reference sources using keyword matching and provides verdicts with confidence scores and citations
This server has a single tool 'verify' with critical definition and implementation gaps. The tool name lacks a clear action verb pattern, the description is generic without WHEN/WHY context, input parameters lack descriptions entirely, and the output schema is not formally documented. The implementation is a naive substring matcher that would not reliably fact-check real-world statements. No input validation, no error handling, no security controls. This is far below production baseline.
Verifies a statement against reference sources and returns a verdict (TRUE, PARTIALLY TRUE, or FALSE) with confidence score and citation
Tool name 'verify' lacks action verb specificity. Should be 'verify_statement' or 'fact_check_statement' to clearly indicate the action and distinguish from other verification operations (verify_email, verify_identity, etc.).
Input parameters 'statement' and 'reference_sources' have no descriptions. LLMs cannot infer what format reference_sources should be (list of URLs? strings? objects?) or constraints on statement length/content.
Output schema is not formally documented. The response object with verdict, confidence, and citation fields is visible only in code, not in an explicit schema definition the LLM can parse.
Tool description is generic and lacks WHEN/WHY context. It does not explain: when to call verify instead of other fact-checking tools, what format reference_sources must be in, or what confidence score thresholds mean.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 22 | 2026-07-28+ | v2 |
Implementation is fundamentally flawed for production use. The algorithm matches statements to references by word overlap (len(word) > 4) without semantic understanding. 'Python is popular for AI' would incorrectly match 'Rubber is used in shoes' if both mention 5-letter words. This is not fact-checking, it is substring matching.
No input validation. The tool accepts any statement and reference_sources list without checking length, format, or type. An LLM could pass statement as an integer or reference_sources as a string, causing runtime errors.
No error handling. If reference_sources is empty, the tool crashes on division by zero (len(matches) / len(reference_sources)). If the list is malformed, there is no recovery guidance for the LLM.
Confidence score is hardcoded (0.5, 0.1, or calculated) with no semantic basis. A confidence of 0.1 for 'FALSE' is misleading, it sounds uncertain rather than definitive.