Loading ToolBench
Top-only academic paper retrieval (CCF / CAS / JCR gated), as a library, CLI and MCP server
Three well-named tools with strong, detailed descriptions (194 - 280 chars each) that explain WHAT, WHEN, and WHY. All parameters have types and descriptions. Input schemas are complete with proper JSON Schema. Output schemas are documented in code. However, no error handling guidance, no enum constraints for filter parameters (ccf_levels, cas_zones accept free-form arrays), and no explicit output schema documentation in the tool definitions themselves. Naming follows verb_noun pattern (search_*, explain_*, preview_*). Descriptions are LLM-optimized and include prerequisites and use cases.
Look up the tier labels (CCF / CAS zone / JCR / WoS index) for one venue. Use it to justify why a paper counts as top-tier, or to check coverage before trusting a filter — an "unknown" answer may mean the venue is not top-tier, or simply that the table covering it is not installed.
Show the query families and topic anchors the planner would search with. Cheap (one LLM call, no source round-trips). Use it when a search returned too little and you want to see whether the terminology was right — most recall problems are a planner that missed the field's own vocabulary.
Search ONLY top-tier academic venues (CCF A/B, CAS zone 1/2, flagship journals). Use this instead of a web search when the user wants the literature a field actually treats as authoritative — a reading list, related work, the state of the art, or "what are the key papers on X". Returns metadata and links, never paywalled full text. Each paper comes back with the reason it qualified, so you can tell the user *why* it counts as top-tier. Also returns a landscape: schools of thought and active teams. Costs one LLM call to plan the search plus several source round-trips, so a query typically takes 30-120s. Ask one well-formed research question rather than many keyword probes.
No enum constraints on ccf_levels and cas_zones parameters. These accept free-form arrays (strings and ints) with no validation. LLMs may pass invalid values like ['X', 'Y'] or [3, 4]. Should declare enums: ccf_levels ∈ {A, B, C}, cas_zones ∈ {1, 2, 3, 4}.
No error handling guidance in tool descriptions. If search_top_papers times out (30 - 120s), returns no results, or hits API limits, the LLM has no recovery path. Should document: 'If no results, try preview_search_plan() to check terminology' and 'Typical latency 30 - 120s; may timeout on complex queries.'
Output schema not formally documented in tool definitions. Code shows response structure (papers array with title, year, venue, tier, citations, authors, url, doi; optional landscape with summary, schools, teams), but this is inferred from implementation, not declared in the schema. LLMs cannot see the expected output structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
No idempotency or retry guidance. search_top_papers calls external APIs (Semantic Scholar, OpenAlex) and may fail mid-stream. No indication whether retrying with the same query is safe or will incur duplicate charges/API calls.
Parameter 'limit' defaults to 20 but accepts up to 50. No guidance on when to use higher limits or performance implications. Should document: 'Higher limits increase latency and token cost; 20 - 30 recommended for most queries.'