Loading ToolBench
MCP server that fetches licensed library PDFs through your own university EZproxy session (University of Michigan defaults)
Server has 6 tools with complete input schemas and descriptions. Naming follows verb_noun convention (auth_status, login, logout, resolve, fetch_pdf, proxy_url). Descriptions are detailed (100-300+ chars) and explain WHEN to use each tool and prerequisites. However, output schemas are not documented, LLMs cannot predict response structure. Error handling is present but inconsistent: some tools return structured error dicts, others may raise exceptions. Parameter descriptions are good but lack explicit constraints (e.g., no enum for live_check/wait_for_login boolean semantics). Risk annotations (READ_ONLY, WRITE, DESTRUCTIVE) are present in metadata but not exposed in tool definitions.
Report whether a U-M library proxy session is active. If a login window is open, this waits for the user to finish signing in before reporting, so you don't have to ask them when they're done. live_check=True then probes the proxy with a headless browser; live_check=False only reports local state.
Fetch a PDF from a publisher URL, DOI, or title. Returns the local path to the saved PDF, or an error with access routes if the fetch fails.
Open a real browser window for the user to sign in to the U-M library proxy (Okta sign-in + Okta Verify). Returns immediately while the window stays open. DO NOT END YOUR TURN AFTER CALLING THIS. Pass the returned `tell_user` text on, then call auth_status (or fetch_pdf, if a paper was already asked for) in the same turn. Both block until the sign-in finishes, so they confirm it for you. The user gets no notification when the window closes, so stopping here leaves them staring at a closed window with no idea whether it worked. Credentials never pass through Claude or this server; only the browser session cookie is kept, in a local profile owned by this user.
Clear the saved library proxy session (deletes the local browser profile). Use when switching users or on a shared machine.
Wrap a publisher URL in the library proxy so it can be opened in a browser with an active session.
Output schemas not documented. LLMs cannot predict response structure (e.g., auth_status returns dict with 'authenticated', 'config_error', 'note' fields, but this is not declared in tool definition). Agents must infer structure from examples.
Boolean parameters (live_check, wait_for_login) lack semantic clarity. Descriptions explain the effect but not the default behavior or when to override. No enum or constraint documentation.
Error handling is inconsistent. auth_status returns structured error dicts (status, code, message), but other tools may raise exceptions. No documented error classification (retryable vs fatal) or recovery guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 70 | 2026-07-28+ | v2 |
Resolve a DOI or a citation/title to article metadata and access routes. Good first step when looking for any paper, article, book chapter, thesis or preprint - including ones that looked inaccessible elsewhere. Returns metadata, a free-copy PDF URL if one exists, the proxied publisher URL, and the library's fulfillment page for the DOI. If the query is a title and the match is uncertain, returns candidates for the user to choose from instead of guessing.
Tool risk annotations (READ_ONLY, WRITE, DESTRUCTIVE) are present in metadata but not exposed in MCP tool definitions. LLMs cannot see which tools are safe to retry or have side effects.
resolve tool returns 'candidates' on uncertain title matches but does not document the structure of candidates or how to disambiguate. LLM must infer next steps.