Loading ToolBench
Evidence-first enrichment engine with search, parsing, analysis, synthesis, and review gates. Exposes enriched and synthesized datasets as MCP resources and tools so that any MCP-compatible AI agent can call into the pipeline directly.
Server has 6 tools with complete schemas and descriptions. Naming follows verb_noun convention (list_, get_, run_, compare_). Descriptions are present and substantive (100-250 chars), exceeding the 10-char minimum. All parameters have type definitions and descriptions. However, several tools share identical parameter sets (entity_id, entity_name, field_name, mode, replay_bundle), creating composition concerns. Output schemas are documented via Pydantic models (ScenarioInfo, SynthesisSummary, etc.), but error handling lacks recovery guidance, tools do not explain what to do if a replay bundle is missing or a pipeline fails. No tool annotations (readOnlyHint, destructiveHint) are present despite all tools being READ_ONLY. Parameter descriptions could be more prescriptive about constraints (e.g., field_name only accepts 'hq_country', but this is stated in description text, not as an enum).
Compare two replay scenarios side-by-side. Returns synthesis summaries for both scenarios and highlights differences in output values and review decisions.
Extract and return all fact claims from a pipeline run. Returns claim count and detailed list of FactClaim objects with sources and confidence scores.
Retrieve the raw contents of a named replay bundle. The bundle contains pre-recorded search results, parsed documents, analysis reports, and synthesis used for zero-API-key replay runs.
Get a concise synthesis summary from a pipeline run. Returns the resolved field value, synthesis reasoning, confidence scores, review decision, and gate reason.
List all available replay scenarios with descriptions. Returns a JSON array of scenario metadata including name and description of what each scenario tests.
Run the full evidence enrichment pipeline for an entity and field. Executes search, parsing, analysis, synthesis, and review gate stages. Returns a complete PipelineRunResult with all intermediate artifacts.
Parameter overloading: Five of six tools accept identical (entity_id, entity_name, field_name, mode, replay_bundle) parameters. This violates single-responsibility principle and forces LLMs to reason about which tool to call when parameters are the same. Consider consolidating into a single 'run_enrichment' tool with a 'result_type' enum (synthesis|claims|full_pipeline).
Missing tool annotations: All tools are READ_ONLY but lack readOnlyHint annotation in schema. This prevents clients from optimizing caching and retry logic. Add 'readOnlyHint: true' to all tool definitions.
Constraint not enforced as enum: field_name parameter description states 'currently only hq_country supported', but parameter is a free-form string. LLMs may pass invalid values. Change to enum: {"type": "string", "enum": ["hq_country"]}.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Error handling lacks recovery guidance: Tools do not document what happens if a replay_bundle is not found, if live mode is blocked by execution policy, or if the pipeline fails. Error responses should include 'Try list_replay_scenarios() to see available bundles' or similar recovery hints.
Output schema incomplete in source: SynthesisSummary model definition is truncated (ends mid-field 'decisio'). Cannot verify all output fields are documented. Ensure full schema is visible and all response fields have descriptions.