MCP RAG Server — semantic search over local documents + web search
mcp-rag-server defines 3 tools with reasonable descriptions and basic input schemas. All tools have names starting with action verbs (knowledge_base_search, web_search, ingest_document). Descriptions are present and provide context on when to use each tool (10 - 1024 character range). However, schemas lack formal type constraints (enums, ranges), parameter descriptions are minimal in JSON Schema form, and output schemas are not documented. Error handling is basic, tools return error strings but do not guide recovery or suggest next steps. The server implements pattern:tool and pattern:tool-description partially, but falls short on pattern:constrained-input and pattern:response-shaper. No tool annotations (readOnlyHint/destructiveHint) are present despite clear security distinctions (knowledge_base_search and web_search are READ_ONLY; ingest_document is WRITE with side effects).
Add a new document to the private knowledge base without restarting. Supports .md, .txt, and .pdf files. The document becomes immediately searchable via knowledge_base_search.
Search the private knowledge base for information from indexed documents. IMPORTANT: ALWAYS call this tool first for ANY question that could be answered by the user's documents. Do not rely on your own knowledge — the knowledge base contains private, authoritative content that you cannot access any other way. If this tool returns "I couldn't find a relevant answer", THEN consider using web_search or your own knowledge.
Search the web for information not found in the private knowledge base. Only use this tool AFTER knowledge_base_search returns no relevant results, or when the user explicitly asks for live/current information from the internet.
No tool annotations (readOnlyHint/destructiveHint) despite clear semantic differences. knowledge_base_search and web_search are read-only; ingest_document modifies state. LLMs cannot infer safety properties from names alone.
Input schemas lack formal constraints (enums, ranges, patterns). top_k parameter on knowledge_base_search has no bounds; title parameter on ingest_document has no format or length constraints. Free-form strings invite invalid LLM submissions.
Output schemas are not documented. knowledge_base_search states it returns 'Document excerpts prefixed with metadata' but the exact format, field names, and structure are not formally specified. LLMs cannot plan downstream processing without knowing the shape of the response.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
Error responses are bare strings with no recovery guidance. Example: 'Error: Knowledge base is not initialized.' does not tell the LLM what to do next (retry? ask user? check logs?). No categorization of errors as retryable vs fatal.
ingest_document accepts filepath as a free-form string with no validation hints. No description of allowed file types (beyond the docstring mentioning '.md, .txt, .pdf'), no path traversal protection hints, no example format. The tool returns a count of chunks but does not return a document_id or reference for downstream queries.
web_search tool requires FIRECRAWL_API_KEY environment variable but the error message is generic: 'Error: Web search failed. Check server logs for details.' LLMs cannot determine whether the failure is transient, a missing credential, or a quota issue.
No pagination support documented for knowledge_base_search results. If the knowledge base contains many matches, no way to iterate through them or cap results. Large result sets risk blowing context window.