Loading ToolBench
AI-powered fraud detection engine using LightGBM models for English and Chinese text analysis with rule-based and ML-based detection
Two tools with significant definition gaps. 'predict' has a reasonable description (127 chars) and documented input schema with types and descriptions. 'check_fraud' has a weak description (67 chars, below 100-char baseline) and minimal schema documentation. Neither tool declares output schemas. Parameter descriptions are present but generic. No error handling guidance, no tool annotations, no idempotency hints. The server exposes internal API details (thresholds, model paths) rather than abstracting them. Naming is acceptable but not verb-forward (predict vs analyze_fraud_risk would be clearer).
Calls the local fraud detection API and returns the detection result as JSON
Analyzes text for fraud using language-specific LightGBM models and rule-based detection. Supports English and Chinese text with automatic language detection.
Missing output schema documentation. Neither tool documents what fields the response contains, forcing LLMs to infer structure from examples.
check_fraud description is 67 characters (below 100-char baseline). It reads as a wrapper description rather than explaining when/why to use it vs predict. Lacks context for LLM tool selection.
No error handling guidance. Tools do not document what errors are retryable, user-fixable, or fatal. check_fraud catches exceptions but returns generic JSON error strings without recovery hints.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Both tools are read-only but this is not declared in the schema, forcing LLMs to infer safety from descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Duplicate/overlapping tools. 'predict' calls the FastAPI endpoint directly; 'check_fraud' wraps it via HTTP. This creates ambiguity, LLMs must reason about which to use.