The XMTP Docs MCP server provides two read-only documentation search tools with complete input schemas and reasonable descriptions. However, several quality gaps limit its rating: (1) descriptions are functional but lack the LLM-optimized depth recommended for agent reasoning (both are ~100 chars, acceptable but minimal); (2) output schemas are entirely undocumented, LLMs cannot see what fields search_xmtp_docs and get_xmtp_doc_chunk return; (3) parameter descriptions are present but lack actionable constraints (e.g., search relevance scoring logic, chunking strategy, format hints); (4) error handling is absent, no guidance on retryability or recovery for network failures, invalid IDs, or empty results; (5) no tool annotations (readOnlyHint, idempotentHint) despite both tools being read-only and idempotent. The code is well-commented internally, but that commentary does not surface in the MCP interface. Input validation via Zod is present and correct. Naming is clear (verb_noun: search_*, get_*) and follows conventions. This is a borderline C/D server, functional for simple search use cases, but insufficient for production agentic systems that require structured response schemas and error guidance.
Retrieves the full content of a specific chunk by ID. AI assistants use this after searching to read the full section.
Searches the documentation and returns matching chunks with previews. AI assistants use this to find relevant documentation sections.
Output schemas entirely undocumented. Neither search_xmtp_docs nor get_xmtp_doc_chunk documents what fields are returned. LLMs cannot plan downstream tool calls or extract specific data from responses without reverse-engineering the implementation.
No error handling guidance. If a search returns no results, the network fails, or an invalid chunk ID is requested, there is no actionable error message or recovery hint. LLMs will not know whether to retry, refine the query, or escalate.
Missing tool annotations. Both tools are read-only and idempotent; they should declare readOnlyHint=true and idempotentHint=true so agents can reason about retry safety and side effects.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 48 | - | v1 |
Parameter descriptions lack actionable constraints. 'limit' specifies range (1-20, default 5) but does not explain truncation behavior or why search results may be incomplete. 'maxChars' default (6000) is arbitrary and undocumented. LLMs cannot reason about when results are truncated.
Tool descriptions lack when/why guidance. 'Searches the documentation and returns matching chunks with previews' tells what it does but not when to call it vs. get_xmtp_doc_chunk, or how search scoring works. 'Retrieves the full content' does not explain the chunking strategy or when to paginate with maxChars.