Loading ToolBench
Three tools with complete schemas and basic descriptions, but significant gaps in parameter documentation and output schema clarity. Tool names follow verb_noun convention (ip_lookup, ip_batch_lookup, ip_validate) which is good. However, parameter descriptions are minimal (e.g., 'IPv4 or IPv6 address to look up' lacks format constraints, valid ranges, or error guidance). No output schema documented, LLMs cannot predict response structure. Error handling returns plain text wrapped in JSON rather than structured error objects with recovery guidance. No tool annotations (readOnlyHint present via Risk field but not in MCP schema). Batch tool lacks per-item error reporting.
Look up geolocation for multiple IP addresses (max 100)
Look up geographic location, ISP, timezone for an IP address
Validate whether a string is a valid IPv4 or IPv6 address
No output schema documented. LLMs cannot predict response structure (fields, types, nested objects). Responses are JSON-stringified text, not structured objects.
Parameter descriptions lack format constraints and validation rules. 'IPv4 or IPv6 address to look up' does not specify valid formats, length limits, or what happens on invalid input.
Error responses return plain text ('Error: <message>') instead of structured error objects with recovery guidance. LLMs cannot determine if errors are retryable or user-fixable.
ip_batch_lookup does not return per-item success/failure. If one IP fails validation, the entire batch response is unclear, LLM cannot determine which IPs succeeded.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
Tool descriptions do not explain when to use each tool or dependencies. E.g., ip_validate description should note 'Call this first to validate before lookup' or 'Use ip_lookup for geolocation; use ip_validate for format checking only'.