Turn any GitHub repository into a searchable knowledge base for AI agents with semantic search, document retrieval, and retrieval weight management
Nancy Brain MCP Server has a moderate quality baseline. All 6 tools are properly named with action verbs (search_, retrieve_, explore_, set_, get_), and all have descriptions present. Input schemas are complete with type definitions and parameter descriptions. However, several critical gaps reduce the score: (1) Output schemas are not documented in the provided code, no return type specifications are visible for any tool, making it impossible for LLMs to plan downstream calls. (2) Error handling guidance is absent, no recovery hints, retryability classification, or actionable error messages are evident. (3) Tool descriptions, while present, are functional but not optimized for LLM prompt-engineering (they lack explicit WHAT/WHEN/WHY framing). (4) The write operation (set_retrieval_weights) lacks confirmation/dry-run capability despite modifying persistent state. (5) Parameter descriptions are present but generic, they do not include constraint details, ranges, or dependency hints. The tools follow good naming conventions and have basic schemas, placing this in the C+/B- range rather than higher.
Explore the document tree structure and list available documents
Get Nancy Brain system status and health information
Retrieve a specific passage from a document by ID and line range
Retrieve multiple document passages in a single request
Search Nancy's knowledge base for relevant documents and code
Set a persistent retrieval preference for this authenticated API key. It affects only future searches made with the same key and never changes another user's ranking. Use it for broadly useful or unhelpful sources, not one-query relevance.
No output schemas documented for any tool. LLMs cannot determine what fields to expect or plan downstream calls. Tools return data but callers must infer structure.
Error handling guidance is absent. No recovery hints, retryability classification, or actionable error messages are visible. Agents will receive bare errors without knowing what to do next.
set_retrieval_weights modifies persistent state but lacks confirmation/dry-run capability. Agents can accidentally override ranking for all future queries without a safety checkpoint.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 65 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 29 | - | v1 |
Parameter constraints are underspecified. e.g. 'threshold' (retrieve calls) has no documented range; 'start'/'end' line numbers lack bounds; 'max_depth' (explore_document_tree) has no min/max. Generic descriptions like 'Search query for the knowledge base' do not guide LLM input.
Tool descriptions lack WHEN/WHY framing. Agents cannot distinguish between search_knowledge_base, retrieve_document_passage, and retrieve_multiple_passages without explicit guidance on use cases and tradeoffs.
Parameter relationships undocumented. anyOf constraint in set_retrieval_weights requires either (weight+doc_id) OR (weight+namespace), but descriptions do not explain the tradeoff or mutual exclusivity.