RAG MCP Server using OpenSearch Serverless - Cloud Ready. Provides tools for managing a knowledge base with document storage in S3 and vector search via OpenSearch Serverless.
Three tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. Tool names follow verb_noun convention (rag_query, rag_add_document, rag_list_documents), which is good. However, descriptions are minimal (34-72 chars, below the 194-char baseline), and critical parameter descriptions are missing or vague. The rag_add_document tool accepts a 'metadata' object with no schema definition. Output schemas are not documented anywhere in the source. Error handling and recovery guidance are absent. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk profiles (WRITE vs READ_ONLY). This server lands in the 'fair' range, definitions exist but lack the depth and structure needed for reliable LLM tool selection.
Add a document to the knowledge base (stored in S3 and indexed in OpenSearch)
List all indexed documents in the OpenSearch knowledge base
Search the knowledge base for information relevant to a query using OpenSearch
Output schemas not documented. LLMs cannot plan downstream calls or extract required fields (e.g., document IDs for chaining). rag_query should document returned fields: document_id, content, score, metadata. rag_add_document should return the s3_key and document_id for reference.
rag_add_document 'metadata' parameter has no schema. Accepts 'object' type with no properties defined. LLMs cannot determine what fields to include. Define schema: {type: 'object', properties: {source: {type: 'string'}, author: {type: 'string'}, ...}, additionalProperties: true}.
Descriptions are too short (34 - 72 chars vs. 194-char baseline). rag_list_documents: 'List all indexed documents in the OpenSearch knowledge base' (59 chars) lacks context on when to call it or what structure it reveals. Expand to: 'List all indexed documents in the OpenSearch knowledge base. Call this first to discover available documents before querying. Returns document IDs, titles, and metadata.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | 2026-07-28+ | v2 |
No tool annotations despite clear risk profiles. rag_query and rag_list_documents are read-only; rag_add_document is destructive (writes to S3/OpenSearch). Add readOnlyHint: true to queries, destructiveHint: true to add_document. This guides agent planning and prevents accidental mutations.
No error handling or recovery guidance. If OpenSearch is unreachable, the agent receives a raw error with no actionable next step. Add error responses: 'OpenSearch unavailable. Retry in 30 seconds or check service status.' For invalid queries: 'Query too broad. Refine with specific keywords or use n_results=1 to test.'