MCP server with FAISS RAG search and GitHub tooling, plus multi-server MCP client with Claude orchestration
Server has 9 tools with consistent naming (verb_noun pattern) and basic descriptions. All tools are READ_ONLY, reducing risk. However, descriptions are generic (avg ~60 chars, well below the 194-char baseline for A+ tools), parameter descriptions lack detail about constraints and formats, and output schemas are not documented. Tools return JSON strings rather than structured objects, forcing LLMs to parse unstructured text. No error handling guidance, no pagination support despite list tools, and no tool annotations (readOnlyHint, etc.). GitHub tools lack enum constraints for 'state' parameters. RAG tools lack documentation of index format and search scoring. Overall: functional but below production baseline.
Get statistics about the current FAISS search index.
Get the contents of a file from a GitHub repository.
Get metadata about a GitHub repository.
List contents of a directory in a GitHub repository.
List issues for a GitHub repository.
List pull requests for a GitHub repository.
Search GitHub repositories.
Descriptions are too brief (avg 45 chars vs 194-char baseline). 'List issues for a GitHub repository' lacks context on when to use vs other tools, what fields are returned, and pagination behavior.
No enum constraints on 'state' parameters (github_list_issues, github_list_pull_requests, github_search_repos). Descriptions say 'open, closed, all' but LLMs may hallucinate invalid values like 'pending' or 'archived'.
Output schemas not documented. Tools return JSON strings (e.g., json.dumps(info.model_dump())) rather than structured objects. LLMs must parse unstructured text to extract fields for downstream calls.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | <=2025-11-25 | v2 |
Search the indexed codebase and documentation using semantic similarity.
Search codebase with optional file path filtering.
No pagination support. github_list_issues, github_list_pull_requests, github_search_repos accept 'limit' but no offset/cursor. Large result sets risk context window exhaustion.
No error handling guidance. If GITHUB_TOKEN is missing, _require_client() raises RuntimeError. LLM receives raw exception; no recovery hint (e.g., 'Configure GITHUB_TOKEN environment variable').