Loading ToolBench
MCP Server for Epic FHIR API integration with OAuth2 authentication and patient data access
The server provides 8 well-named read-only tools for FHIR patient data access. All tools follow the verb_noun naming convention (get_*, search_*) and have clear, actionable descriptions (72-97 chars, within the 10-1024 baseline). Input schemas are complete with proper type definitions and enums where appropriate. However, there is NO documented output schema for any tool, the server has response formatters (format_patient_response, format_bundle_response, format_resource) visible in src/tools.py, but these are internal implementations, not exposed as part of the tool definition. LLM callers have no way to know what fields to expect. Additionally, parameter descriptions are minimal (e.g., 'FHIR patient ID' lacks context on what an ID looks like or when a lookup is needed). The search_patients tool lacks required field constraints, it has optional parameters with no guidance on which combinations are valid or recommended. Error handling is not visible in the tool definitions themselves; recovery guidance is absent. All tools are READ_ONLY (good security posture), but no tool annotations (readOnlyHint) are declared in the MCP schema to communicate this formally.
Get patient demographics and basic information by FHIR ID
Get patient's allergies and adverse reactions
Get patient's medical conditions, diagnoses, and health problems
Get patient's vaccination and immunization history
Get patient's current and past medications and prescriptions
Get patient's clinical observations (labs, vitals, etc.)
NO OUTPUT SCHEMA DOCUMENTED for any of the 8 tools. Tools return formatted strings via format_patient_response() and format_bundle_response(), but the tool definition in src/tools.py does NOT include an outputSchema or return type declaration. LLMs cannot plan downstream calls or extract structured data without knowing the response shape.
Example values embedded in parameter descriptions (e.g., 'FHIR patient ID (e.g., erXuFYUfucBZaryVksYEcMg3)'). Rubric explicitly prohibits this: LLMs tend to reuse example values literally. Replace with format constraints or remove the example and rely on the description text.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) declared, even though all 8 tools are READ_ONLY. Formal annotations per the MCP spec improve agent reasoning about side effects and retry safety.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 58 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Get patient's medical and surgical procedures
Search for patients by demographics (name, birthdate, gender)
Minimal parameter descriptions lack context on constraints and dependencies. Examples: 'patient_id' describes WHAT it is but not FORMAT, RANGE, or when it's needed; search_patients has 4 optional parameters with no guidance on which combinations are valid or recommended. Rubric requires 'describe the expected format, range, and allowed values directly in the parameter description.'
Tool descriptions lack context on when to use a tool vs a similar alternative. Example: get_patient vs search_patients, when is lookup preferred? What if I only know a partial name? No guidance in the descriptions. Rubric requires: 'Answer: What does it do? When should the LLM call it instead of a similar tool? What does it return?'
No error handling or recovery guidance visible in tool definitions or code samples. What happens if a patient_id is invalid? If search returns no results? If the FHIR server is down? Rubric requires: 'Error responses must tell the LLM what to do next.' and 'Categorize errors as retryable, user-fixable, or fatal.'
search_patients lacks pagination support. Tools that return lists SHOULD accept page/offset and limit parameters and return a total count. search_patients currently has no such parameters, risking large unbounded result sets.
Ambiguous tool descriptions leave scope unclear. Examples: 'medications', current only, or all historical? 'procedures', medical AND surgical? Incomplete docstrings force LLMs to guess. Rubric: 'A tool description must answer: What does it do? When should the LLM call it instead of a similar tool? What does it return?'