Loading ToolBench
Local OpenAlex database with 284M+ works, abstracts, and semantic search. MCP server for Claude integration enabling full-text search across academic papers.
Server has 4 tools with basic descriptions and parameter schemas. However, critical gaps limit production readiness: (1) Tool definitions appear inferred from file paths rather than explicit registration code visible in source; (2) Parameter descriptions are present but generic, many lack actionable constraints (e.g., 'query' has no length/format guidance, 'limit' has no min/max bounds); (3) Output schemas are completely undocumented, no response structure defined for any tool; (4) Error handling is absent, no guidance on recovery, retryability, or user-fixable vs. fatal errors; (5) No tool annotations (readOnlyHint, idempotentHint) despite all tools being read-only. Naming is clear (search, search_by_id, status, enrich_ids) and follows verb_noun convention. Descriptions exist but are brief (50-150 chars) and lack WHEN/WHY context for LLM selection.
Enrich OpenAlex IDs or DOIs with full metadata. Use this after search() to get detailed metadata for papers. The search() tool returns basic info (title, authors, year, source). This tool adds: abstract, concepts, is_oa, oa_url, etc.
Search for academic works by title, abstract, or authors. Uses FTS5 full-text search index for fast searching across 284M+ papers. Supports FTS5 query syntax: AND, OR, NOT, "exact phrases".
Get detailed information about a work by OpenAlex ID or DOI.
Report local OpenAlex database status — path, work count (should be ~284M), FTS5 index size, and access mode (local / remote). Use when the user asks "is the OpenAlex db ready?", "how many papers do I have?", "check openalex-local status", "is the FTS index built?", or before running a search to verify the db is usable.
Output schemas completely undocumented. No response structure defined for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., does search() return paper_id, doi, or both? What fields does enrich_ids() add?).
Parameter constraints missing. 'limit' has no min/max bounds (unbounded integers invite absurd values). 'query' has no length limit or format guidance. 'offset' lacks validation. Descriptions state defaults but not ranges.
No error handling guidance. Tools lack recovery hints (e.g., 'If query returns no results, try broader terms or check status() first'). No indication of retryability or user-fixable vs. fatal errors.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
Tool annotations missing. All 4 tools are read-only (no state modification), but readOnlyHint is not declared. Agents cannot infer safety/idempotency without explicit hints.
Descriptions lack WHEN/WHY context. 'Get detailed information about a work' does not explain when to call search_by_id vs. search(). 'Enrich OpenAlex IDs' does not clarify that this is a follow-up to search(). LLMs struggle with tool selection without explicit guidance.