Plugin-based RAG system for knowledge management with document processing, indexing, and LLM-powered query answering
This server has 27 tools with significant quality gaps. Most tools lack proper input schemas (only 3-4 fields visible per tool, no type definitions beyond 'string', 'file', 'object', 'array', 'integer', 'boolean'). Descriptions are present but generic (avg ~80 chars, well below the 194-char baseline for A+ tools). Critical issues: (1) No output schemas documented anywhere, LLMs cannot plan downstream calls. (2) Parameter descriptions are minimal ('Username', 'Password') without constraints, formats, or validation guidance. (3) No error handling guidance in any tool description. (4) Sensitive tools (delete_user, delete_tool, remove_mcp_server) lack confirmation/dry-run patterns. (5) Authentication tools (login, logout) expose session state management without clear permission gates. (6) Tool composition is poor, upload_document + query + process_document overlap; register_tool + add_mcp_server + connect_to_server form unclear chains. (7) No idempotency guarantees stated. (8) Naming is inconsistent: 'add_user' vs 'delete_user' (verb_noun), but 'get_current_user' vs 'get_permissions' (inconsistent object naming). Per-tool analysis: upload_document, query, login, logout, get_current_user, get_permissions, add_user, delete_user, register_tool, delete_tool, get_tools, get_tool_briefs, add_mcp_server, remove_mcp_server, get_mcp_servers, connect_to_server, disconnect_from_server, build_bm25_index, search_index, update_index, clean_index, process_document, extract_keywords, analyze_query, assess_content_systematicity, rerank_results, health_check all score 30-45 individually due to missing schemas and weak descriptions.
Add a new MCP server connection
Add a new user to the system (admin only)
Analyze user query to determine intent and extract keywords
Assess how systematic and complete content is using LLM
Build BM25 full-text search index for all documents
Clean up the index by removing references to non-existent documents
Establish connection to an MCP server
Delete a registered MCP tool from the system
No output schemas documented for any tool. LLMs cannot infer what fields to expect or plan downstream tool calls. E.g., query returns results but structure is unknown; get_tools returns a list but field names are undocumented.
Parameter descriptions are minimal (1-3 words) without constraints, formats, or validation guidance. E.g., 'Username' lacks length limits, character restrictions, or case sensitivity rules. 'Query' lacks max length enforcement (stated as 2000 chars but not validated in schema).
Destructive tools (delete_user, delete_tool, remove_mcp_server) lack confirmation or dry-run patterns. No description warns of irreversibility. Agents can delete resources without safeguards.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 35 | 2026-07-28+ | v2 |
Delete a user from the system (admin only)
Disconnect from an MCP server
Extract keywords from text using LLM
Get information about the currently authenticated user
Get list of configured MCP servers and their connection status
Get permission configuration for all roles
Get brief information about all available tools
Get list of available MCP tools
Check the health status of the RAG system
Authenticate user with username and password
Logout current user and destroy session
Process a document for RAG: extract text, analyze content, and assess systematicity
Stream-based RAG query interface that searches indexed documents and generates answers using LLM
Register a new MCP tool in the system
Remove an MCP server connection
Rerank search results using LLM to improve relevance
Search the document index using keyword matching
Update the search index with a new plugin
Upload and process documents (PDF, DOCX, TXT, MD, XLSX, XLS, CSV, JSON, HTML) for RAG indexing
No error handling guidance in any tool description. Tools do not explain what to do on failure (retry? ask user? fatal?). E.g., login fails but description does not say 'Try search_users() if username unknown' or 'Retryable: rate limit'.
Tool composition is unclear. upload_document, process_document, and query overlap in document handling; register_tool, add_mcp_server, and connect_to_server form ambiguous chains. No descriptions explain when to use which tool or what data flows between them.