Loading ToolBench
NYPL Digital Collections MCP. New York Public Library Digital Collections — 900k+ digitised items: photographs, maps, manuscripts, prints & ephemera. Free API token required.
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation, output schema clarity, and error handling guidance. Tool names are verb-based (search, item) which is good, but descriptions lack actionable context for LLM selection. Parameters like 'limit' and 'page' have descriptions but lack explicit constraints (e.g., '1-100' is mentioned but not formalized). The _apiKey parameter violates secret-injection best practices by appearing in tool signatures. No documented output schemas, pagination guidance, or error recovery paths. Code shows input validation (clamp, reqStr) but error messages are generic.
Fetch full details for one NYPL Digital Collections item by id — an NYPL item uuid (from search).
Search the NYPL Digital Collections collection by keyword. Returns matching items with ids (pass an id to item), titles, creators/sources, dates and links.
API key exposed as tool parameter (_apiKey). Credentials must never appear in tool signatures, use server-side secret injection via environment variables or vault. Agent traces log all parameters, leaking secrets into logs and prompt history.
No documented output schemas. LLMs cannot plan downstream calls or extract required fields without knowing what the API returns. Both tools lack response structure documentation (e.g., what fields does search return? Is there pagination metadata?).
Error handling lacks recovery guidance. Code throws generic errors ('NYPL Digital Collections: key rejected/missing') but does not tell the LLM what to do next or categorize errors as retryable vs fatal. No guidance on partial failures or alternative actions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
Parameter constraints not formalized in schema. 'limit' description says '1-100, default 20' but schema lacks minValue/maxValue. 'page' lacks minimum constraint. LLMs cannot reliably enforce constraints from text descriptions alone.
Pagination not fully documented. 'search' accepts 'page' and 'limit' but response structure (total count, has_more, next_page) is not documented. LLMs cannot determine when to stop paginating or how many results exist.