Loading ToolBench
Drive the AI co-scientist research engine from Claude Code, Codex CLI, and any MCP client
Strong tool naming (verb_noun pattern) and comprehensive parameter descriptions. All 9 tools have clear, detailed descriptions (100-300 chars) explaining WHAT, WHEN, and consequences. Input schemas are well-structured with proper types and enums. However, output schemas are not documented, LLMs cannot predict response structure for downstream chaining. Error handling guidance is absent. Tool composition is sound: each tool has one responsibility, and session_id/hypothesis_id are consistently named across tools.
List a session's hypotheses ranked by Elo, with title, summary, state, matches played and dedup cluster. Bodies are omitted; use hypothesis_get.
One hypothesis in full: statement, detailed text, citations, Elo history, every review recorded against it, and its tournament matches. Read this before reviewing or ranking it.
Add a hypothesis to a session, in the same table Generation and Evolution write to. It enters as a draft, gets reviewed, and then competes in the tournament like any other. Set parent_ids when it derives from existing hypotheses, which is what makes it an Evolution result rather than a fresh idea. Recording is what makes a hypothesis real: an idea only described in chat is never reviewed, ranked, or seen by the meta-review.
Abort a session for good. The main loop exits at its next check.
List sessions newest first, with status, hypothesis count, top Elo and budget spent. Use this to find a session id the user refers to by topic.
Pause a running session. Workers drain and the loop sleeps until resumed.
Output schemas not documented. Tools like session_list, session_status, hypotheses_list, and hypothesis_get return complex nested structures (hypothesis counts, Elo spread, match history, reviews, citations) but LLMs cannot predict field names or types. This breaks downstream tool chaining and forces agents to guess at response structure.
No error handling guidance. Tools like session_start (which runs for minutes to hours) and hypothesis_record (which writes to shared tables) lack recovery instructions. What should the agent do if session_start times out? If hypothesis_record fails due to dedup conflict? If a session_id is invalid?
Pagination parameters present but no total count or next_cursor returned. session_list and hypotheses_list accept limit/offset but descriptions do not state whether results are exhaustive or if more exist. LLMs cannot determine when to stop paginating.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | <=2025-11-25 | v2 |
Resume a paused or interrupted session. The engine reclaims expired task leases and continues where it stopped. Returns immediately; poll session_status.
Start a new research session from a natural-language goal. The engine runs the full Generation, Reflection, Elo tournament, Evolution and Meta-review loop in a detached process; this returns as soon as the session id exists. Poll session_status for progress. It runs for minutes to hours, so agree the scale with the user first: call estimate_cost.
Progress for one session: status, task counts by state, hypothesis counts by state, matches played, Elo spread, budget spent, and whether the final overview exists yet. This is the polling tool.
Destructive operations (session_abort, hypothesis_record with parent_ids) lack confirmation or dry-run support. An agent could irreversibly abort a multi-hour research session without explicit user consent.
Parameter 'strategy' in hypothesis_record uses enum but description does not explain what each strategy means (literature vs debate vs combine). LLMs cannot reason about which strategy to choose without semantic guidance.