Loading ToolBench
Universal document evaluation engine — MCP server for Claude subagent orchestration with multi-agent document assessment, ontology-based alignment, and structured evidence scoring.
Brain in the Fish defines 15 specialized evaluation tools with explicit input schemas (via schemars::JsonSchema derives in server.rs). All tools have descriptions, but most are 50-150 characters, adequate but minimal for LLM disambiguation. Input parameters are typed (String, usize, f64, Vec, Option) with descriptions. However, output schemas are NOT documented, a critical gap for tool composition. Tool naming uses domain-specific prefixes (eval_*, eds_*) which aid discovery but are less action-verb-oriented than the baseline (e.g., 'eval_criteria' is unclear: does it fetch, create, or load?). No explicit error handling guidance, recovery paths, or validation constraints visible. No output field documentation means downstream tools cannot plan parameter assembly. The rubric baseline for A+ is 100% param descriptions + documented return types; this server has the former but completely lacks the latter.
Record a challenge to an agent's score using counter-evidence, supporting structured debate and multi-round evidence reconciliation.
Feed structured evidence extracted by a subagent into an agent's argument graph for a specific criterion, accumulating evidence for scoring.
Load OWL Turtle RDF (from LLM-based Phase 1 decomposition) into the graph store, enabling semantic reasoning over structured document ontology.
Compute structured argument-graph scores for an agent across criteria using the Evidence-Driven Scoring (EDS) gate function, optionally constrained to a single criterion or lot.
Align document sections to evaluation criteria using ontology-based structural matching and keyword overlap, returning mappings and gaps.
Generate a challenge prompt for one agent to challenge another agent's score on a specific criterion, supporting debate-style multi-round evaluation.
No documented output schemas for ANY tool. LLMs cannot predict field names or types for downstream composition. A tool returning 'agent_panel' might embed agent data as flat fields or nested objects, LLMs must guess, forcing trial-and-error calls.
Inconsistent parameter naming: some tools use 'criterion_id' (string), others imply usize indices. No explanation of when to use names vs indices. LLMs conflate these without clear guidance.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | <=2025-11-25 | v2 |
Load or auto-detect an evaluation framework (generic, academic, policy, clinical, legal) based on intent or explicit framework name.
Load and display cross-evaluation history and trends from the memory store, including all previous evaluations and their metadata.
Ingest a PDF file and extract its content into the session document, with an evaluation intent to guide downstream assessment.
Extract or generate evidence predictions from document sections using rule-based extraction or LLM-mode prompts for subagent credibility assessment.
Record a score awarded by an agent for a criterion, including justification, evidence references, and identified gaps.
Generate a scoring prompt for a specific agent and criterion, providing context (document sections, related criteria) for subagent evaluation.
List all pending scoring tasks (agent–criterion pairs) for subagent assignment, optionally filtered by agent or criterion.
Spawn an evaluator panel (3–5 domain-specific agents plus a moderator) based on the evaluation intent and framework.
Modify a document section's content and re-evaluate against the criteria, supporting counterfactual analysis and impact assessment.
No enum constraints on mode/framework parameters. 'eval_criteria' accepts framework as free-form string (should be enum: generic|academic|policy|clinical|legal). 'eval_predict' mode omits enum (should be prompt|extract). LLMs hallucinate invalid values.
No error handling guidance. No recovery paths (e.g., 'if agent_index out of bounds, call eval_spawn first'). No validation constraint documentation (when does eval_ingest fail?). Agents cannot self-correct.
Tool naming lacks action verbs. 'eval_criteria' does not start with get/list/create/update (unclear action). 'eval_what_if' is metaphorical (should be 'counterfactual_reeval' or 'test_document_variant'). 'eds_feed' (feed into what?). Naming should lead with verb.
Incomplete parameter type visibility. eval_align criterion_id is marked optional but type (string? usize?) unclear. eval_predict section_index type is usize but mode enum values are not enumerated. Code snippet truncated, full type definitions not provided.
No indication of tool dependencies or call sequence. LLMs must infer: eval_spawn before eval_score_prompt? eval_criteria before eval_spawn? Document prerequisites (if you have no panel, call eval_spawn first).
Descriptions are terse (60 - 100 chars) relative to rubric baseline (194 chars median for A+ tools). Many lack context on WHEN to call vs similar tools. E.g., eval_score_prompt vs eval_challenge_prompt, when does LLM choose one?
No pagination or result limits documented. eval_history may return years of evaluations; eval_scoring_tasks could list hundreds of pending tasks. No mention of cursor, offset, limit parameters or max result counts.