Loading ToolBench
MCP server wrapping Sakana AI Scientist v2 for autonomous research loops
Single tool 'research_ideate' has a detailed description (180+ chars) and comprehensive input schema with 5 parameters, all typed and described. However, the tool exhibits critical composition and naming issues: it combines ideation, tree-search execution, and hypothesis extraction into one operation (violates single-responsibility pattern). The name 'research_ideate' is vague, does not start with a clear action verb (should be 'generate_research_hypotheses' or 'ideate_research'). Output schema is not documented in the tool definition; the code shows internal extraction logic (_extract_hypotheses) but no declared return type for the LLM. Error handling is absent, no guidance on what happens if the AI Scientist service fails, times out, or returns malformed ideas. No validation of model string, max_num_generations bounds, or num_reflections constraints. The tool accepts a 'workshop_file' path parameter without sanitization, creating a path-traversal risk.
RESEARCH_IDEATE — Generate 5 research hypotheses via Progressive Agentic Tree Search. PORTMANTEAU PATTERN RATIONALE: This server exposes a small, high-signal tool surface aligned with the Sakana v2 loop. Ideation is kept as a dedicated tool because it is frequently invoked and should remain stable while internal tree-search implementations evolve.
Tool name 'research_ideate' is vague and does not start with a clear action verb. Should be 'generate_research_hypotheses' or 'ideate_research_hypotheses' to signal intent clearly.
Output schema is not documented. The tool extracts hypotheses internally but does not declare the return type (list of {id, title, hypothesis} objects) in the tool definition. LLMs cannot plan downstream actions without knowing the response structure.
Tool combines ideation, tree-search execution, and hypothesis extraction, three distinct concerns. Should split into separate tools (e.g., 'generate_ideas', 'run_tree_search', 'extract_hypotheses') so agents can compose them independently.
No error handling or recovery guidance. If AI Scientist service fails, times out, or returns malformed data, the tool provides no actionable error message. Code includes a fallback (_fallback_hypotheses) but does not document when/why it triggers.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | <=2025-11-25 | v2 |
Path parameter 'workshop_file' is not validated or sanitized. Accepts arbitrary file paths without bounds checking, creating a path-traversal vulnerability. Should validate against a whitelist or restrict to a specific directory.