MCP server for answering questions about AcmeCompany IT tickets using RAG with Q Business and Bedrock, integrated with Perplexity for general knowledge
Single tool 'answer_question' has critical definition gaps. Input schema visible but minimal: only a 'question' parameter with basic string type and description. No output schema documented. Description is present (28 chars) but falls short of ideal 50-200 char range for LLM optimization. No parameter constraints (enums, ranges, validation rules). Error handling not evident in source. Tool appears to call external AWS services (Q Business, Bedrock) but no documented prerequisites, failure modes, or recovery guidance visible in the tool definition.
Get answer to a question using RAG with Q Business and Bedrock
No output schema documented. Tool returns 'str' (response from Bedrock) but LLM has no visibility into response structure, confidence scores, source citations, or error indicators.
Tool description is only 28 characters ('Get answer to a question using RAG with Q Business and Bedrock'). Falls below recommended 50-200 character range. Lacks guidance on WHEN to use it, prerequisites (credentials, Q Business setup), or expected latency.
Single parameter 'question' lacks constraints. No description of valid question types, length limits, expected format, or examples. No guidance on questions that will/won't work (e.g., only IT ticket queries indexed in Q Business).
No visible error handling or recovery guidance in tool definition. If Q Business search returns no results, or Bedrock inference fails, or credentials are invalid, the LLM has no documented recovery path. Source shows error-prone credential loading and external service calls but tool definition offers no hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 43 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 37 | - | v1 |
Credentials loaded from environment (TVM values file, DEFAULT_EMAIL, region) but tool definition exposes no parameters for these. If credentials are invalid or missing, error messaging will be opaque to the LLM. Credential injection pattern not documented.
Tool calls external AWS services (Q Business, Bedrock) with no documented latency, timeout, or rate-limit behavior. Agents calling this tool may hang indefinitely if services are slow. No pagination or result-limiting guidance visible.
Tool name 'answer_question' is generic. Better names: 'search_it_knowledge_base', 'query_q_business_index', or 'retrieve_it_ticket_info' would clarify that this is domain-specific (IT tickets) and RAG-backed, not a general QA tool.