Loading ToolBench
Model Context Protocol server for verified access to jw.org content
The server has 4 explicitly registered tools with complete JSON Schema definitions and non-empty descriptions. Naming follows verb_noun convention consistently (search_content, get_article, get_scripture, get_cache_stats). However, several quality gaps reduce the score: (1) Parameter descriptions lack actionable detail, e.g., 'filter' enum values are listed but no guidance on when to use each; (2) No output schemas documented, responses are TextContent wrappers but the actual data structure (how search results are formatted, what fields article responses contain) is not specified; (3) Error handling is generic ('Error:', 'Unexpected error:') with no recovery guidance; (4) No pagination support on search_content despite potentially large result sets; (5) Tool descriptions omit dependencies, e.g., 'get_article' assumes the URL comes from search_content but doesn't say so. Per-tool analysis follows.
Retrieve full article content from a JW.Org URL. Returns the article text with paragraphs and scripture references.
Get cache statistics including hit rate and entry count.
Get scripture text by reference (e.g., 'John 3:16', '1 Thessalonians 5:3'). Returns the scripture text and reference.
Search JW.Org content including articles, videos, publications, audio, and scriptures. Extracts meaningful search terms from natural language queries.
No output schemas documented for any tool. Response structure for search results, article content, scripture text, and cache stats is not specified. LLMs cannot plan downstream operations or extract fields they need.
Parameter descriptions lack actionable constraints and validation rules. 'filter' lists enum values but does not explain content type differences. 'translation' parameter accepts a code ('nwtsty') with no documentation of valid codes or format.
Error handling is generic and non-actionable. All errors return TextContent with plain text messages ('Error:', 'Unexpected error:') with no guidance on recovery, categorization (retryable vs. user-fixable), or suggested next steps.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 42 | - | v1 |
search_content has no pagination support (no offset/limit fields in response, no next_cursor). Maximum limit is 50 items, which may be insufficient for large result sets, and agents cannot iterate through pages.
Tool descriptions lack dependency hints and composition guidance. 'get_article' does not mention that URLs typically come from search_content. Users may not know the expected workflow or may call tools in wrong order.
Example values ('John 3:16', '1 Thessalonians 5:3') embedded in descriptions instead of enforced as formal constraints (regex patterns, enums, or format declarations). LLMs may reuse examples literally rather than adapt to actual input.