Async, typed RAG assistant for operational documentation. Answers procedural questions from an approved local Markdown SOP corpus via MCP tools and streams typed SSE events.
TraceRAG defines 2 tools with clear, well-written descriptions (194 - 250 chars each, within baseline 34 - 392 range). Both tools have explicit input schemas with typed parameters and descriptions. Naming follows verb_noun convention (search_procedures, get_procedure). However, output schemas are not documented in the visible code, return types are inferred from type hints (list[RetrievedChunk], ProcedureDocument|None) but no structured schema is exposed to the MCP client. Error handling is minimal: no guidance on what happens when a search returns empty, when a document_id is invalid, or how to recover. No parameter constraints (enums, ranges) are declared. The tools are read-only and well-scoped, but lack the production-grade error recovery and output documentation expected of A-grade tools.
Fetch a full procedure document by its document id (for example 'SOP-004'). Use this only when a search result needs more surrounding context than the chunk provided. Returns the document metadata and full Markdown content, or None when the document id is unknown — never invent documents.
Search the approved Markdown SOP corpus for chunks relevant to a procedural question. This is the mandatory first step before answering: retrieve candidate chunks, then base every factual statement on the returned content. Returns at most `limit` scored chunks with their source document id, title, section, content, and overlap score. Chunks with no match are not returned.
Output schemas not documented. Type hints exist (list[RetrievedChunk], ProcedureDocument|None) but no structured schema is exposed to the MCP client. LLMs cannot plan downstream operations or extract fields without explicit schema documentation.
No error handling guidance. search_procedures returns empty list on no match, but no description of failure modes or recovery steps. get_procedure returns None for unknown document_id, but no guidance on how to discover valid IDs or what to do next.
No parameter constraints. 'limit' parameter has no min/max bounds (default 3, but unbounded). 'query' and 'document_id' accept any string with no validation hints or format constraints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | 2026-07-28+ | v2 |
RetrievedChunk and ProcedureDocument models not visible in provided source. Cannot verify that response fields match downstream tool parameters or that chaining IDs are included.
No tool annotations (readOnlyHint, idempotentHint). Both tools are read-only and idempotent, but this is not declared in the schema, forcing LLMs to infer safety from descriptions alone.