MCP server for searching and reading AWS Karpenter documentation
Two tools with clear naming, reasonable descriptions, and complete input schemas. Both tools follow verb-noun naming convention (search_documentation, read_documentation) which aligns with LLM expectations. Input schemas are well-structured with types and descriptions. However, output schemas are not documented in the visible code, parameter descriptions lack constraint information (e.g., format examples for 'path'), and error handling guidance is minimal. The server implements sensible defaults (limit=10, max=50) and validates input early, but lacks actionable error recovery messages. Both tools are READ_ONLY (safe), reducing risk. Overall definition quality is solid but missing output documentation and detailed constraint specifications that would push this toward 80+.
Read the full content of a specific Karpenter documentation page.
Search AWS Karpenter documentation by keyword query.
Output schemas not documented in tool definitions. LLMs cannot plan downstream steps without knowing what fields are returned (e.g., search results return 'path', 'relevance', 'section'?). This forces the LLM to guess result structure.
Parameter 'path' in read_documentation lacks format/constraint information. Description says 'e.g. docs/concepts/scheduling.md' but does not specify: required file extension? path depth limits? security constraints (prevent ../traversal)? LLMs infer constraints from descriptions, missing details invite invalid inputs.
Error handling responses are generic ('No results found for query') and do not guide recovery. Per the rubric, error messages must suggest next steps: e.g., 'No results for "scaling", try broader term "scale" or search section="concepts" first.' Current implementation provides no recovery guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 61 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 57 | - | v1 |
Parameter 'section' in search_documentation is optional but description lists 'common sections' as examples (docs, guides, concepts, reference, troubleshooting). Per the rubric, examples in descriptions invite literal reuse by LLMs. Should use an enum constraint instead.
Pagination not explicitly addressed in search_documentation. If a real search returns >10 results, does the tool return a 'next_cursor' or 'has_more' flag? Undocumented pagination behavior forces the LLM to guess whether to retry with offset or assume results are complete.