MCP server for healthcare provider discovery and doctor information lookup with A2A agent orchestration support
Single tool (list_doctors) with basic structure but significant gaps in definition quality. The tool has a reasonable name starting with a verb (list_), an adequate description (168 characters, within baseline range), and parameters with type definitions and descriptions. However, the schema lacks critical production-grade features: no output schema documentation, no pagination support despite returning lists, no error handling guidance, and parameters are marked as optional (not required) without rationale. The tool is READ_ONLY (safe), but the server lacks any mechanism for handling invalid input beyond a simple error dict return. The description includes example output in the docstring ('Example: [...]'), which violates the pattern guidance to avoid example values that LLMs may reuse literally. Input validation is minimal (checks for at least one param) but doesn't validate format of state codes or city names. The tool definition is directly visible in code (examples/mcpserver.py), so no inference penalty applies, but the overall quality is below production baseline (median 45-55 for typical servers; this lands at lower end due to missing output schema, pagination, and error guidance).
This tool returns a list of doctors practicing in a specific location. The search is case-insensitive.
No output schema documented. Tool returns list[dict] but LLM cannot predict field names (name, specialty, address, etc.) or structure for downstream chaining.
No pagination support. Tool returns all matching results without limit or cursor, risking context window exhaustion if dataset grows. Rubric baseline: paginated-result pattern required for list tools.
Parameters marked optional without defaults or rationale. Both state and city are optional (not required: false), yet the tool's only validation is 'please provide a state or a city' error dict. This is ambiguous to LLMs, should they always pass at least one?
Example output in docstring ('Example: [......]'). Rubric guidance: avoid example values in descriptions; LLMs may reuse them literally rather than adapting to context. Use formal schema and constraints instead.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
Error response structure inconsistent. Tool returns [{'error': '...'}] (dict in a list), but normal response is list[dict]. This inconsistency may confuse LLM parsing. Errors should be categorized (retryable, user-fixable, fatal) per recovery-guide pattern.
No input validation on state code format. State is accepted as any string; tool silently handles invalid codes (e.g., 'ZZ', 'Ca') via case-insensitive comparison. Should validate state against known codes (50 US states + territories) and return actionable error.
No limit parameter or result count in response. If doctors.json grows to 10,000+ records, a state query could return all matching doctors without trimming. Rubric: results should be capped at 20-50 with pagination.