Knowledge Base RAG Agent with semantic search - MCP server for knowledge retrieval based on document summaries with vector embeddings
The Knowledge Base MCP server provides 6 tools with generally clear naming and reasonable descriptions. Most tools follow verb-noun conventions (search_summaries, fetch_document, save_knowledge, list_categories, update_document, delete_document). Descriptions are present and substantive (range: 100-250 chars), explaining WHAT the tool does and WHEN to use it. However, critical gaps exist: (1) No tool annotations (readOnlyHint/destructiveHint/idempotentHint) despite having read-only, write, and destructive operations clearly labeled in risk metadata, this information should be in the tool schema itself; (2) Error handling descriptions are largely absent, tools describe happy paths but not failure modes or recovery guidance; (3) Output schemas are documented in toolkit.py dataclasses but not explicitly exposed in the tool definitions themselves; (4) Parameter descriptions lack some specificity around constraints (e.g., min_relevance is described as '0.0-1.0' but the description should note it filters results below the threshold, not above); (5) No confirmation mechanism for destructive operations (delete_document). The server follows good composition patterns, each tool does one thing, tool names are unambiguous, and responses should chain well. Naming is consistent and action-oriented. Average tool-level definition quality is adequate but not exemplary.
Delete a document from the knowledge base by ID. Removes the document, its summary, and associated markdown file.
Fetch the full content of a document by ID. Use this after summary search to retrieve the complete document for detailed reading.
List all available knowledge categories and document counts. Use this to understand the structure of your knowledge base and what categories are available before searching.
Save new knowledge to the knowledge base. This stores a document and marks it for summary generation in the background. The function returns immediately - file writing and summarization happen asynchronously.
Search the knowledge base by document summaries using semantic search. Use specific, descriptive queries for best results. Add context and key concepts rather than vague terms.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) in schema despite risk metadata clearly labeling operations. Pattern:tool-annotation requires this metadata to be discoverable by agents.
Destructive operation (delete_document) lacks confirmation/dry-run mechanism. Per pattern:confirmation-request, irreversible operations should support a confirmation step to prevent agent mistakes.
Error handling guidance is absent from tool descriptions. Tools do not explain failure modes, which errors are retryable, or how to recover. Per pattern:recovery-guide, error responses must tell the LLM what to do next.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 65 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 58 | - | v1 |
Update document content and regenerate summary. Only updates the content field - title, category, and tags remain unchanged.
Output schemas are defined in dataclasses (SummaryResult, DocumentResponse, OperationResponse, CategoriesResponse) but not explicitly surfaced in the MCP tool registration. LLMs cannot see these schemas without code inspection.
delete_document parameter 'document_id' description says 'Document ID to delete' but does not explain consequences or require explicit confirmation. No warning that deletion is permanent.
search_summaries min_relevance parameter description could be clearer. It states '0.0-1.0' but should explicitly note it filters OUT results below the threshold, and may need guidance on typical values (0.3 for broad, 0.5 for focused).