An MCP server template for Retrieval-Augmented Generation (RAG) that indexes documents and retrieves relevant chunks using vector embeddings.
Three tools with basic verb-noun naming (list_, search_, retrieve_) but significant quality gaps. All tools have descriptions, but they are generic and lack actionable context (avg ~50 chars, well below the 194-char baseline). Input schemas are present but minimal: list_sources_tool has empty input {}; search_documents_tool and retrieve_chunks_tool have only 'question' (string) and 'top_k' (integer, default 3) with minimal parameter descriptions. No output schemas documented. No error handling guidance, no parameter constraints (e.g., top_k range), no indication of what fields are returned. Tools lack idempotency hints, permission declarations, and recovery guidance. The handlers.py implementation shows basic functionality but no validation, rate limiting, or error classification.
List all supported source files in the data folder.
Retrieve structured chunks for a user question.
Search indexed documents using a user question and return matching sources/chunks.
No output schemas documented. LLMs cannot predict what fields to expect (e.g., does search_documents return 'chunks' or 'results'? Is 'sources' a list of strings or objects?). Forces LLMs to guess and wastes tokens parsing unstructured responses.
Parameter descriptions are missing or trivial. 'top_k' has description 'Number of top results to return' but no range constraints (min/max). LLMs may pass absurd values (top_k=10000) causing timeouts or API errors.
No error handling or recovery guidance. If OpenAI API fails (rate limit, invalid key), handlers.py will raise an unhandled exception. LLM receives a stack trace with no actionable next step.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 39 | 2026-07-28+ | v2 |
Tool descriptions are under 50 characters and lack context. 'List all supported source files in the data folder' does not explain WHEN to call this vs search_documents, or what structure is returned. Baseline is 194 chars.
No pagination support. If data folder contains 1000 documents, search_documents returns all top_k chunks in a single response, risking context window exhaustion. No limit enforcement or next_cursor documented.