Loading ToolBench
Lightweight, on-demand MCP server for structured methods extraction and reproducibility heuristics on academic papers.
methods-mcp demonstrates solid tool design with clear naming (verb_noun pattern), comprehensive descriptions (avg 180 chars), and well-structured schemas. All 8 tools have input schemas with type definitions and parameter descriptions. Output schemas are documented via Pydantic models (HealthReport, PaperMetadata, MethodsStructured, etc.). Error handling includes recovery guidance (e.g., 'Couldn't fetch full text for repo discovery' with fallback). Main gaps: no tool annotations (readOnlyHint/destructiveHint), limited parameter constraints (prefer enum is good, but model param lacks bounds), and some descriptions could be more LLM-optimized (e.g., extract_methods is 280 chars, slightly verbose). Composition is strong, tools chain well (metadata → full text → methods/repo/summary). No security issues detected (no secrets in params, environment-based injection via ANTHROPIC_API_KEY).
Heuristic, no-clone reproducibility assessment of a GitHub repo. Returns a verdict (likely-reproducible / partial / unlikely / insufficient-info) plus weighted signals: README quality, dependency files, fixture data, notebooks, figure-generating scripts, recent maintenance, license. No code is downloaded or executed. Set GITHUB_TOKEN to raise the API rate limit.
Extract a structured methods object from a paper. Pulls full text, isolates the methods-relevant section(s), then asks Claude (default: claude-sonnet-4-6) to fill out a Pydantic schema with steps, reagents, equipment, and analyses. Validation-enforced — invalid responses trigger one repair attempt before raising. Requires ANTHROPIC_API_KEY in the server's environment.
Return best-effort full text + section split for a paper. prefer: one of "auto" (default), "html" (force ar5iv HTML — arXiv only), or "pdf". Sections are normalised to lowercase canonical names: abstract, introduction, methods, experiments, results, discussion, conclusion, limitations, related_work.
Discover the code repo (if any) associated with a paper. Strategy: scan the paper's full text and abstract for github.com URLs; fall back to Papers With Code lookup for arXiv papers.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All tools are read-only but lack explicit annotation for LLM safety reasoning.
Parameter 'model' in extract_methods and summarize_paper lacks validation bounds. Should document valid Claude model identifiers and default behavior more explicitly.
extract_methods description (280 chars) is verbose and could be condensed to 150-200 chars for better LLM token efficiency. Mentions 'Validation-enforced, invalid responses trigger one repair attempt' which is implementation detail, not user-facing behavior.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | <=2025-11-25 | v2 |
Resolve an arXiv ID/URL, bioRxiv URL, DOI, or generic URL to canonical metadata. For arXiv inputs this hits the arXiv export API to fetch title/authors/abstract. Other sources return the resolved URL with empty bibliographic fields (cheap by design).
Return a small health report — confirms the server is alive and configured.
Resolve a paper + extract methods + find its code repo + assess reproducibility — in one call. Returns a `MethodsReproReview` object. Read `narrative` first — it's a plain-English summary of everything retrieved, with every numeric score echoed in context. Then: `status`: `"ok"` (all sub-steps succeeded), `"partial"` (some failed — see `errors`), or `"empty"` (couldn't even resolve the input paper). `metadata`, `methods`, `code_repo`, `repro_assessment`: the structured sub-results. Any step that failed contributes `null` for its key; `null` means **unavailable**, not zero. `errors`: list of `{step, error_type, message, hint}` — one entry per failed sub-step, with an actionable hint where the failure pattern is recognised.
Generate an LLM summary of a paper in one of three modes. abstract = 2-3 sentences close to the authors' framing. tldr = one-line takeaway. exec = executive summary (what / found / why-it-matters).
methods_repro_review description is very long (500+ chars) and buries key info. Should lead with 'Composite tool: resolves paper, extracts methods, finds code repo, assesses reproducibility in one call' then detail the response structure.
Error handling in find_code_repo catches Exception broadly and logs warning, but returns a CodeRepo with null fields rather than structured error. Should use pattern:recovery-guide to guide LLM on retry strategy.