CodeVault has 8 well-named tools with complete input schemas and descriptions. Tool names follow verb_noun convention (search_code, index_project, ask_codebase). All tools have descriptions (avg ~60 chars, within baseline). Input schemas are properly typed with Zod validation. However, output schemas are not documented, LLMs cannot predict response structure. Parameter descriptions are minimal (e.g., 'Search query' lacks format/constraint guidance). Error handling is structured (MCPErrorPayload with code/type/message) but lacks recovery guidance. No tool annotations (readOnlyHint/destructiveHint). Missing: pagination guidance for search results, idempotency guarantees, and per-parameter constraint documentation.
Ask a question and get LLM-synthesized answer with code citations
Get code chunk by SHA
Get project statistics
Index a project for semantic search
Search code semantically using embeddings
Search code and return full code chunks
Update project index
Output schemas not documented. LLMs cannot predict response structure (fields, types, pagination). Rubric baseline: 100% of A+ tools have documented return types.
Parameter descriptions lack constraint guidance. E.g., 'limit' has no min/max bounds; 'provider' enum values not explained; 'path' lacks validation rules. Rubric: 'Specify minimum and maximum for numeric parameters.'
No tool annotations (readOnlyHint/destructiveHint/idempotentHint). LLMs cannot determine which tools are safe to retry or have side effects. Rubric: 'tool annotations (readOnlyHint/destructiveHint/idempotentHint)' are current patterns.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | <=2025-11-25 | v2 |
Activate a context pack for scoped search
Error responses lack recovery guidance. formatMcpError() returns code/type/message but no 'suggestion' for most errors. Rubric: 'Error responses must tell the LLM what to do next.'
Pagination not documented for search results. search_code returns 'limit' param but no 'offset', 'cursor', or 'total_count' guidance. Rubric baseline: tools returning lists should accept page/offset and return total count.