An OpenReview manuscript search and analysis service with MCP protocol support
Server defines 5 tools with generally clear names and consistent schema structure. Tool descriptions are present and moderately detailed (range 60-180 chars), exceeding the 20-char minimum but below the production baseline of 194 chars. All tools include input schemas with types and descriptions. However, no output schemas are documented, the server returns TextContent wrapped results, but the underlying data structure (JSON, fields, types) is not specified. This forces LLMs to infer output structure from tool names alone. Parameter descriptions are present for all inputs and explain constraints (enums, patterns, limits), but lack actionable guidance on error recovery or dependencies between tools. The export_papers tool (WRITE risk) has no confirmation step or dry-run option, violating the confirmation-request pattern. Error handling is minimal, generic exception wrapping without classification or recovery guidance. Overall, the definitions are functional but fall short of production-grade standards for agentic tool design.
Export papers to JSON files for further analysis and code development
Fetch all accepted papers from a specific conference venue and year
Fetch all papers published by a specific user identified by email
Search for papers by keywords within specific conference venues
Find an OpenReview user profile by email address
No output schemas documented for any tool. All tools return TextContent, but underlying data structure (fields, types, nesting) is not specified. LLMs must infer what fields to extract and how results chain to downstream tools.
export_papers is a destructive write operation (creates files, downloads PDFs) with no confirmation step, dry-run option, or capability to undo. Violates confirmation-request pattern. Agents could accidentally write to disk or consume bandwidth without user approval.
Error handling is minimal. All tools wrap exceptions in generic TextContent responses without classification (retryable vs fatal), recovery guidance, or actionable next steps. E.g., if search_user returns no profile, the LLM is not told to try search_papers instead.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 60 | <=2025-11-25 | v2 |
| 2026-03-09 | C | 64 | - | v1 |
Tool descriptions are 65-123 chars, below the production baseline of 194 chars. Descriptions lack context on WHEN to use the tool vs similar tools, dependency hints, or expected outcomes. E.g., search_papers description does not explain how it differs from get_conference_papers or explain match_mode semantics.
Parameter constraints are declared in schema but not summarized in descriptions. E.g., export_papers.max_papers has a range (1-10) in schema but the description does not explain why it is capped or what 'maximum number of papers to download' means operationally.
No tool accepts natural identifiers for lookups beyond email. The search_user tool accepts email (good), but if an LLM has only a user name, there is no way to look them up. No search_users() tool exists to discover users by partial name before calling search_user.
get_user_papers and get_conference_papers accept a 'format' parameter (summary vs detailed) but do not explain the data difference or when to choose each. LLMs will guess, potentially fetching unnecessary data.