A Retrieval-Augmented Generation chatbot platform with document management, conversation tracking, and optional MCP (Model Context Protocol) integration for GitLab
17 tools with basic schemas and descriptions, but significant gaps in parameter documentation and error handling. Most tools have descriptions (10-50 chars, below the 50-200 char baseline for LLM optimization). Parameter descriptions are sparse or missing entirely. No output schemas documented. Error handling is absent. Tool naming is clear and verb-based, but composition issues exist (ask_question conflates RAG and agent modes). This is a typical community server with functional but incomplete definitions.
Ask a question in a conversation with streaming response. If the collection has MCP configured (e.g., GitLab integration), uses the AgentService with tool calling. Otherwise uses regular ChatService for RAG-only responses.
Create a new collection.
Create a new conversation.
Delete a collection and all its documents.
Get aggregate metrics for dashboard.
Get public application configuration. Returns configuration values that are safe to expose to the frontend, such as the current LLM model name.
Get conversation history.
Output schemas not documented. LLMs cannot infer what fields to expect from tool responses, forcing them to guess at downstream field names and risk broken tool chains.
Descriptions are too short (10-50 chars vs. baseline 50-200 chars). LLMs cannot determine when to select a tool or why it differs from similar tools. E.g., 'Create a new conversation' lacks context on prerequisites or when to use it.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
Get metrics for all messages in a conversation.
Get metrics for all conversations.
Get daily metrics summary.
Get hourly metrics statistics.
Health check endpoint.
List all collections.
List conversations in a collection.
List documents in a collection.
Update a collection's configuration.
Upload a document.
Parameter descriptions missing or minimal. E.g., 'collection_id' has description 'UUID of the collection' but no guidance on how to obtain it or what happens if invalid. Parameters like 'top_k' in ask_question lack range constraints (e.g., 1-100).
ask_question conflates two distinct modes (RAG-only vs. agent with tool calling). Description says 'If the collection has MCP configured... uses AgentService... Otherwise uses ChatService', this is a composition issue. Split into ask_question_rag and ask_question_with_tools, or document the mode selection clearly.
No error handling guidance. Tools lack descriptions of failure modes, recovery steps, or actionable error messages. E.g., delete_collection has no warning about irreversibility or confirmation step, violating the confirmation-request pattern for destructive operations.