Loading ToolBench
MCP server for HEOR Studio — turns a reviewed study spec into deterministic SAS + SQL for MarketScan. Deterministic tools by default; an LLM extract tool activates only when an API key is present.
HEOR Studio MCP has 9 well-intentioned tools with domain-specific purpose (medical code search, study spec validation, code generation). Naming is verb-forward and clear (search_codes, validate_spec, generate_code). Descriptions are substantive (100 - 250 chars) and explain WHEN to use each tool. However, input schemas are partially visible but lack complete type definitions for complex objects (e.g., validate_spec's 'spec' parameter is typed as 'object' with no nested schema). Output schemas are not documented. Error handling is minimal, no recovery guidance or actionable error messages. Tool composition is sound (search → validate → generate → read pipeline), but parameter descriptions could be more explicit about constraints and dependencies.
Extract a StudySpec from a protocol document using Claude. Requires ANTHROPIC_API_KEY. Returns a spec_id; call validate_spec to review and store it.
Deterministically generate study code from a stored spec. REFUSES unless the spec is ready, every code is verified, AND signoff=true — this enforces human review. Returns a manifest of file handles (path, bytes, sha256), never file contents; read files with get_artifact.
Read a single file from a generated artifact, line-paged so large programs stay within MCP output limits. Use the paths from generate_code's manifest.
Fetch the MarketScan native schema map (tables, columns, roles, grain).
Fetch the StudySpec JSON schema (heor://schema/study-spec) for validation and IDE autocomplete.
List available MarketScan database models (CCAE, MDCR, MDCD) and their characteristics.
Complex object parameters lack nested schema definitions. validate_spec accepts a 'spec' object but no nested schema is visible, LLMs cannot infer the required structure of StudySpec.
Output schemas are not documented. Tools return results (e.g., generate_code returns 'manifest of file handles') but the response structure is not formally specified. LLMs cannot plan downstream calls without knowing what fields to expect.
Error handling lacks recovery guidance. No tool description explains what to do if validation fails, code generation is refused, or verification errors occur. Errors should guide the LLM to the next step.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 69 | 2026-07-28+ | v2 |
Verify generated code against a synthetic dataset
Look up candidate codes to help build a code list. system='icd10cm' searches ICD-10-CM diagnosis codes (NLM Clinical Tables); system='rxnorm' searches drug names (RxNav). Returns candidates only — the analyst still verifies each code before use.
Validate a StudySpec (conforming to the heor://schema/study-spec resource) and store it, returning a spec_id. Reports readiness and how many codes remain unverified. Also used to SAVE an edited spec after the analyst confirms code lists — resubmit the spec with verified:true on confirmed codes. Verification flags you set are preserved.
Parameter constraints are implicit. generate_code's 'tag' parameter has a regex pattern but the description does not explicitly state the constraint. LLMs cannot read JSON Schema patterns, they rely on description text.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). The protocol supports these hints to clarify tool safety, validate_spec and generate_code are destructive but unmarked.