Loading ToolBench
A Model Context Protocol server for accessing National Bureau of Economic Research (NBER) working papers and managing a local feed database
The NBER CLI MCP server exposes 10 tools with basic descriptions and parameter schemas. However, the evaluation reveals significant gaps in description quality, parameter documentation, and output schema specification. Tool names follow a clear verb_noun pattern (get_, set_, add_, remove_, refresh_), which is positive. However, most parameter descriptions are minimal (10-30 characters), and output schemas are not documented in the provided source. The server implements READ_ONLY and WRITE risk annotations at the tool level, but lacks error recovery guidance, input validation constraints, and detailed parameter constraints. Of the 10 tools, 4 are READ_ONLY and 6 are WRITE, indicating this is a stateful application wrapper. Tools like 'get_paper' have the side effect of marking papers as read, which should be documented more explicitly. Parameter descriptions exist but are sparse, e.g., 'NBER paper ID (e.g., w12345)' repeats across multiple tools without explanation of the format constraint itself. Overall, this server represents a functional but mediocre integration that would benefit from richer descriptions, explicit output schemas, and error recovery patterns.
Adds a user-defined tag to a paper
Retrieves the desktop application configuration including feed refresh interval, font size, config path, database path, and log directory
Retrieves a paginated list of NBER paper feed items with optional limit and offset parameters
Retrieves detailed information about a specific NBER paper by its ID, including full metadata and marks it as read
Retrieves the current application settings
Refreshes the NBER paper feed by fetching the latest papers and metadata, returning statistics on the refresh operation
Removes a tag from a paper
Output schemas not documented. Tools return unspecified structures. LLMs cannot plan downstream calls or extract required fields. E.g., get_paper, refresh_feed, get_config lack documented return types.
Parameter descriptions are minimal and lack constraint details. E.g., 'NBER paper ID (e.g., w12345)' repeats across tools but does not specify format rules, valid patterns, or why the format matters. Parameters like 'tag' in add_paper_tag have 11-char descriptions.
Side effects not documented. get_paper marks the paper as read automatically, but the description does not mention this destructive side effect. Agents cannot distinguish safe retrieval from state mutation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2026-07-28+ | v2 |
Renames an existing tag for a paper
Saves updated application settings including feed refresh interval and font size
Sets the read status of a specific paper
No error recovery guidance. Tools lack descriptions of failure modes, retryability, and actionable error messages. E.g., if add_paper_tag fails due to invalid paper_id, the agent has no guidance on how to recover.
No input validation constraints. Parameters like 'limit' in get_feed state '1-200, default 100' but lack minimum/maximum JSON Schema constraints. Parameter 'detail_font_size' in save_settings must be 14, 16, or 18, should be an enum, not free-form integer.
Pagination not complete. get_feed accepts limit and offset, but does not document whether total_count is returned or whether there is a next_cursor for cursor-based pagination. Agents cannot safely iterate large result sets.
Tool descriptions under 50 characters for several tools (get_config, get_paper, add_paper_tag, etc.). Descriptions should be 50 - 200 chars to provide sufficient context for LLM tool selection.