Verify, repair, and enrich BibTeX references across academic metadata providers
Bibverify provides 4 well-defined tools with clear verb-based names and reasonable descriptions. All tools have documented input schemas with type information and parameter descriptions. However, several gaps limit quality: (1) output schemas are not explicitly documented in the source, forcing LLMs to infer response structure; (2) parameter descriptions are functional but minimal (averaging ~30-50 chars), lacking actionable constraints and format guidance; (3) no error handling guidance beyond ToolError exceptions; (4) descriptions are adequate (60-90 chars each) but lack strategic details about when/why to use each tool; (5) no tool annotations (readOnlyHint) despite all tools being read-only operations. The tools are well-composed (each does one thing) and follow appropriate naming conventions. Parameter typing is present but sparse.
Fetch one DOI through Crossref and return a BibTeX entry.
Compare two BibTeX-like entries and return field-level changes.
Return the effective provider order for a reference.
Verify the configured BibTeX file and return a structured summary.
Output schemas not documented in source code. LLMs cannot plan downstream operations without knowing what fields to expect from tool responses.
Tool descriptions are brief (55-65 chars) and lack strategic guidance. No description explains WHEN to use each tool vs. others, or what downstream operation typically follows.
Parameter descriptions are sparse and generic. E.g., 'The DOI to fetch' lacks format constraints (e.g., 'standard DOI format: 10.xxxx/yyyy'). No guidance on what constitutes a valid DOI or error recovery if invalid.
No tool annotations present. All 4 tools are read-only operations and should declare readOnlyHint:true to signal safety to LLMs, preventing unnecessary confirmation steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2025-06-18+ | v2 |
Error handling is minimal. ToolError is raised for workspace violations, but descriptions lack recovery guidance (e.g., 'Check workspace_root configuration' or 'Try expanding the permitted path range').
Optional parameters (key, config_file, entry) have required=false but lack clear defaults. No description states: 'If omitted, [default behavior].' LLMs must guess defaults.