MCP server that performs evidence-aware retrieval and citation of records based on query matching, ranking results by relevance and providing citations
Single tool with a compound name ('evidence_aware_retrieval') that violates verb-first conventions. Schema is visible but parameters lack descriptions. Tool description exists but is generic (98 chars, below 194-char baseline). No parameter descriptions in the actual schema. No output schema documentation. No error recovery guidance. Input validation exists in code but error messages are generic. No support for pagination despite the tool accepting a list. Missing idempotency and timeout handling. Overall, this is a minimal implementation that satisfies basic tool registration but falls well short of production-grade agent patterns.
Retrieves and ranks records based on query term matching, computing relevance scores and providing evidence citations
Tool name does not start with clear action verb. 'evidence_aware_retrieval' is ambiguous, does it retrieve, search, rank, cite, or all four? Should be 'search_records', 'rank_records', or 'search_and_rank_records' (though compound names should be split).
Input parameters 'records' and 'query' lack descriptions in the schema. The schema shows type and description fields exist, but descriptions are missing entirely.
No output schema documented. Tool returns a dict with 'query', 'matches', 'count', 'status' but the response structure is not formally documented in the MCP tool definition. LLMs cannot plan downstream actions without knowing the response shape.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 31 | 2026-07-28+ | v2 |
Error handling does not guide recovery. Line 'response = {"status": "error", "error": str(exc)}' returns raw exception text (e.g. 'records must be a list'). Current approach gives no actionable guidance.
Tool accepts 'records' (a list) but provides no pagination, limit, or offset parameters. If records list is large, tool returns all ranked results, risking context window exhaustion.
Tool description (98 chars) is below the 194-char average for A+ tools and lacks key context. It does not state WHEN to use this tool vs. other search/retrieval alternatives, what preprocessing it does (term extraction, scoring algorithm), or output field meanings. Descriptive baseline p10=34, p90=392; this is below average.
No idempotency guarantees documented. Tool re-ranks on every call; results should be deterministic given the same inputs. If the agent retries due to a transient error, does the ranking change? This must be explicit.
No timeout handling. If records list is massive (e.g. 100k items), the nested loop (for i, record in enumerate(records)) could hang. Tool should document max records accepted or timeout after N seconds.