Agentic Paper-to-Code Research Assistant — RAG + LangChain Agent + MCP + GitHub. Provides paper search, GitHub repository discovery, file listing, code search, and file reading capabilities.
Five tools with clear verb-noun naming (search_paper, find_repository, list_repository_files, search_code, read_file). All have descriptions (avg ~150 chars, within baseline 194). However, input schemas are visible but lack critical structure: no type declarations beyond 'string', no enums for constrained inputs (e.g., branch names), no output schemas documented. Parameter descriptions exist but are minimal (avg ~40 chars vs baseline 72). No error handling guidance, no pagination support for list_repository_files despite returning potentially large trees. Tool descriptions state WHAT but not WHEN to use or prerequisites. Missing idempotency hints, rate-limit guidance, and recovery paths.
Find the GitHub repository associated with a research paper. Search GitHub for repositories that implement or reference a specific research paper based on its title, authors, and keywords.
List all files in a GitHub repository. Returns a structured file tree showing the repository layout.
Read the contents of a specific file from a GitHub repository.
Search for code in a GitHub repository. Find specific functions, classes, algorithms, or patterns in the repository source code.
Search the uploaded research paper for relevant information. Use this tool when the user asks questions about the research paper, its methodology, contributions, results, or any specific section. The tool retrieves the most relevant passages from the paper using advanced hybrid retrieval and cross-encoder reranking.
No output schemas documented. LLMs cannot plan downstream calls or extract structured data. search_paper returns 'context' (string?), find_repository returns formatted text, list_repository_files truncates at 200 items without pagination metadata.
No pagination support. list_repository_files silently truncates at 200 items without returning total_count, next_cursor, or indicating truncation. Large repos will be incomplete.
Parameter descriptions are minimal (avg 40 chars). 'query' in search_paper lacks format guidance. 'branch' in list_repository_files and read_file lacks explanation of default behavior. 'extension' in search_code lacks examples (py, js, etc.).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | <=2025-11-25 | v2 |
No error handling guidance. GitHub API errors (404, 403, rate limits) are caught but return generic strings ('not found', 'rate limit'). No recovery hints (e.g., 'Try with fewer keywords' or 'Retry in 60 seconds').
No input validation or constraint documentation. search_code 'path' and 'extension' parameters lack format specs. find_repository accepts free-form 'keywords' with no guidance on length or format. No enums for constrained values.