Loading ToolBench
Three tools with clear verb-noun naming (lookup_*, search_*) and basic descriptions. All tools have input schemas with type definitions and parameter descriptions. However, descriptions are minimal (10-50 chars, well below the 194-char baseline), lack context on WHEN to use each tool, and do not explain dependencies between tools. No output schema documentation visible. Error handling returns structured ErrorResponse but lacks recovery guidance. Missing parameter constraints (e.g., state must be valid abbreviation; no enum for NpiEnumerationType in search tools). No pagination support despite API accepting limit/skip. Tools are read-only (good security posture) but descriptions do not state this explicitly.
Lookup a provider by NPI number
Search for individual providers by name and state
Search for organizations by name and state
Descriptions are too brief (10-50 chars vs. 194-char baseline). Lack WHEN/WHY context. 'Lookup a provider by NPI number' does not explain when to use lookup_npi_number vs. search_individual_providers, or what the output contains.
No output schema documented. LLMs cannot predict what fields SearchResponse/ErrorResponse contain, forcing them to guess how to extract NPI, name, credentials, or other provider details from results.
State parameter lacks enum constraint. Code defines NpiStateAbbreviation enum (50 valid values) but does not expose it to LLM. LLM may pass invalid state codes, causing API errors with no guidance on valid options.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2026-07-28+ | v2 |
No pagination support exposed. API accepts limit (1-200) and skip parameters but tools do not expose them. Large result sets will be truncated silently or blow context window.
Error responses lack recovery guidance. ErrorResponse returns error codes (e.g., 'VAL001') but does not tell LLM what to do next or how to fix invalid input.