Loading ToolBench
Self-Growing Research Intelligence system with 40+ tools for research discovery, hypothesis generation, writing, code analysis, knowledge management, and self-improvement
Polymath has 31 tools with significant definition gaps. Most tools have descriptions (good), but parameter schemas are inconsistent, many lack type definitions or descriptions. Tool naming is reasonable (verb-noun style: search_papers, find_gaps, generate_hypothesis), but several tools have overlapping responsibilities (e.g., search_papers, semantic_search, hybrid_search all search; generate_hypothesis appears twice with different signatures). Output schemas are not documented. Error handling is absent, no recovery guidance or actionable error messages visible. The server is STDIO-only, which is a hard transport cap. Overall, definitions are functional but fall short of production quality.
Add a paper and its entities to the Knowledge Graph.
[stable] Check knowledge base health: recency, coverage gaps, code-paper linkage, citability rate. Returns health report + action items.
Find papers that discuss both concepts.
Execute a Cypher query on the Knowledge Graph.
[stable] Deep literature hunt across all sources (Polymath corpus, Europe PMC, arXiv, bioRxiv, GitHub). Returns top 20 papers + code repos with evidence summaries.
Discover papers from OpenAlex (250M+ works).
Duplicate tool definitions: generate_hypothesis appears in both polymath_v11/server.py and neo4j_server.py with different signatures (research_area vs topic). LLMs cannot disambiguate and will select unpredictably.
Three overlapping search tools (search_papers, semantic_search, hybrid_search) with unclear distinctions. Descriptions do not explain when to use each. LLMs waste reasoning cycles deciding between them.
Generic getter tools (get_stats, get_graph_stats, get_corpus_stats) have minimal descriptions (<50 chars). LLMs cannot determine what data each returns or when to call them instead of alternatives.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields. Responses are opaque.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
[beta] Propose expansion targets based on strategy: fill_gaps, follow_citations, trending. Returns expansion plan for approval.
[beta] Find analogous solutions from unexpected domains. Abstracts problem to structural pattern and maps solutions.
Find paths between two concepts or methods.
[alpha] Hunt for datasets across GEO, SRA, Zenodo, GitHub, paper supplements. Uses keyword heuristics - direct API integration pending.
Find evidence spans that support a claim using NLI.
[stable] Detect research gaps: orphan concepts, time gaps (no recent work), method gaps (technique not applied to domain). Returns ranked opportunities.
Find papers similar to a given text snippet.
Find method-concept combinations that appear only once (research gaps).
[stable] Generate cross-domain hypotheses using structure mapping. Finds unexplored concept bridges and scores by novelty (corpus-based) x feasibility x testability.
Generate a polymathic hypothesis by finding cross-domain connections.
Get statistics about the RAG corpus.
Get current Knowledge Graph statistics.
Get papers specifically about a topic.
Get current system statistics.
Run a Cypher query against the Neo4j knowledge graph.
Search across papers + code + graph with RRF fusion.
Ingest a code file into the knowledge base.
Ingest a PDF into the knowledge base.
Search indexed code repositories.
Search indexed papers with semantic + lexical hybrid search.
Search the paper corpus by semantic meaning.
[stable] Surface unexpected but potentially useful connections. Returns surprising concept bridges you might not think to look for.
[beta] Analyze publication velocity, emerging methods, rising authors in a field. Returns trend report with visualizations.
[beta] Validate a hypothesis against the corpus. Decomposes into claims, scores each with heuristics, identifies gaps. NLI scoring coming soon.
[alpha] Track a lab/author's recent publications, methods, datasets, and collaborators. Currently searches by name only - external API integration pending.
No error handling or recovery guidance visible. Tools return failures with no actionable next steps for the agent.