Loading ToolBench
Citation management and academic paper processing utility with tools for converting DOIs to BibTeX, extracting metadata from various sources, formatting and validating citations, and searching academic databases
21 tools with inconsistent quality across naming, descriptions, and schemas. Naming is generally verb-first and action-oriented (doi_to_bibtex, convert_multiple, extract_from_doi, etc.), which is good. However, descriptions are present but often lack actionable detail about WHEN to use each tool vs. similar alternatives, and many lack guidance on error recovery. Input schemas are visible and mostly well-typed, but output schemas are either undocumented or inferred from tool names. No explicit error handling guidance, no tool annotations (readOnlyHint/destructiveHint), and no structured error messages to guide LLM recovery. The codebase shows reasonable parameter descriptions in docstrings, but these are not consistently surfaced in the tool registration metadata visible in the MCP interface.
Convert multiple DOIs to BibTeX entries with rate limiting between requests. Takes list of DOIs and returns list of BibTeX entries, excluding failed conversions.
Remove duplicate BibTeX entries based on DOI or citation key. DOI comparison is prioritized as more reliable than key comparison.
Detect duplicate entries in BibTeX file based on DOI and citation key matching. Returns list of duplicate groups with severity levels.
Convert a single DOI to BibTeX format using CrossRef content negotiation API. Cleans DOI input (removes URL prefixes) and returns BibTeX string, converting @data type entries to @misc if needed.
Extract metadata from arXiv ID using arXiv API. Returns structured metadata including title, authors, year, and DOI/journal reference if the paper was published.
Extract metadata from DOI using CrossRef API. Returns structured metadata including title, authors, year, journal, volume, issue, pages, publisher, and URL.
Output schemas not documented. Tools like extract_from_doi, extract_from_pmid, search, and validate_entry return complex structured objects but the MCP interface does not publish their return schemas. LLMs cannot predict the structure of results and must guess at field names for chaining.
No error recovery guidance. Tools call external APIs (CrossRef, NCBI, arXiv, Google Scholar) and can fail with network errors, rate limits, or invalid IDs. Error messages (if any) do not guide LLMs on what to try next (retry, fallback tool, lookup data).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 57 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 53 | - | v1 |
Extract metadata from PubMed ID using NCBI E-utilities API. Returns structured metadata including title, authors, year, journal, volume, issue, pages, and DOI if available.
Fetch detailed metadata for list of PubMed IDs in batches of 200. Returns list of metadata dictionaries with title, authors, journal, year, volume, issue, pages, DOI, and abstract.
Fix common BibTeX formatting issues including page range hyphens (single to double), page prefix removal, DOI URL cleaning, and author separator normalization.
Format a single BibTeX entry with standard field ordering, proper alignment, and BibTeX syntax. Fields ordered according to standard convention (author, title, journal, year, etc.)
Format entire BibTeX file with options for deduplication, sorting, and fixing common issues. Can output to new file or overwrite original.
Identify the type of academic identifier (DOI, PMID, arXiv ID, or URL) and return cleaned identifier. Supports multiple URL formats including DOI.org, PubMed, and arXiv URLs.
Convert metadata dictionary to BibTeX entry string. Automatically determines entry type and formats fields appropriately.
Convert PubMed metadata to BibTeX @article format with automatic citation key generation based on first author, year, and PMID.
Convert Google Scholar metadata to BibTeX format with automatic citation key generation and entry type detection based on venue.
Parse BibTeX file and extract all entries with their type, citation key, and fields. Uses regex pattern matching to identify @entrytype{key, fields...} structures.
Search Google Scholar for publications using scholarly library with optional proxy support. Returns list of metadata dictionaries with title, authors, year, venue, citations, and URLs.
Search PubMed using NCBI E-utilities API with query, date range, and publication type filters. Returns list of PMIDs matching search criteria.
Sort BibTeX entries by specified field (key, year, author, or title) in ascending or descending order.
Validate single BibTeX entry against required and recommended fields, check year format, DOI format, page range format, and author separators. Returns list of errors and warnings.
Verify DOI resolves correctly and retrieve metadata from CrossRef API. Returns tuple of (is_valid, metadata) with title, year, and authors if available.
No tool annotations. The server advertises risk levels (READ_ONLY, WRITE) but does not use readOnlyHint, destructiveHint, or idempotentHint in tool metadata. LLMs cannot detect which operations are safe for dry-run, confirmation, or retry.
Metadata conversion tools (metadata_to_bibtex, metadata_to_bibtex_scholar, metadata_to_bibtex_pubmed) have overlapping names and similar responsibilities. LLMs cannot easily distinguish when to use each variant. No description explains the differences or when to prefer one over the others.
Parameter descriptions lack specifics. Examples: sort_entries accepts sort_by: 'key', 'year', 'author', 'title' but these are in free-form string descriptions, not declared as enum constraints. search accepts year_start/year_end but does not specify valid ranges or format. fix_common_issues says 'Fix common BibTeX formatting issues' but does not list which issues (page hyphens, DOI URLs, author separators, etc.).
Descriptions are functional but generic. Most tool descriptions are 60 - 100 characters and state WHAT the tool does but not WHEN or WHY to use it. 'Convert metadata dictionary to BibTeX entry string' lacks context: when would an LLM choose metadata_to_bibtex vs. metadata_to_bibtex_scholar vs. metadata_to_bibtex_pubmed? No guidance on prerequisites or prerequisites.
Pagination not implemented. search (Google Scholar) and search_pubmed return up to 50 and 100 results respectively, but neither tool accepts limit, offset, or pagination parameters. If an LLM needs to loop over results or handle large result sets, it has no cursor or offset to resume from.
Idempotency not declared. format_file can write to the original file or a new output. Calling it twice with the same inputs may produce different results if the file is modified between calls. No mention of idempotency or whether the operation is safe to retry.