AI-powered document search engine — hybrid BM25 + vector retrieval with Claude answer synthesis
CodeSight exposes 6 tools via FastAPI HTTP server. All tools have names and descriptions present, and input parameters are typed with JSON Schema. However, descriptions are inconsistent in quality and depth, several parameters lack descriptions entirely, and output schemas are not documented in the tool definitions. Error handling is minimal, tools do not guide LLMs on recovery paths. The server stores API keys server-side (good security posture) but lacks tool annotations (readOnlyHint/destructiveHint) despite having mixed risk profiles. Per-tool analysis: search and ask have solid query/top_k/file_glob/source parameters but output schema is undocumented; index lacks a description of what force_rebuild actually triggers and no warning about destructive behavior; status, health, and config are minimal stubs with no parameters and trivial descriptions.
Ask a question about indexed documents; LLM synthesizes answer with source citations
Get public configuration (auth requirement and LLM backend)
Check server health and configuration status
Index or rebuild the document index from the documents directory
Search indexed documents using hybrid BM25 + vector retrieval
Check the indexing status and repository metadata
Output schemas are not documented. Tools return results (Answer, SearchResult, IndexStats, RepoStatus) but the response structure is not visible in tool definitions. LLMs cannot plan downstream field extraction or chaining.
index tool description is cryptic: 'Index or rebuild the document index from the documents directory'. Does not explain what force_rebuild=true does, what data is destroyed, or that it is destructive. Missing destructiveHint annotation.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). The index tool is destructive and requires marking; search/ask/status/health/config are read-only and should be marked as such for agent safety.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 66 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 46 | - | v1 |
status, health, and config tools have trivial descriptions (50 chars or less) and no parameter documentation. status description is 'Check the indexing status and repository metadata', what fields are returned? health is 'Check server health and configuration status', what does success look like?
Error handling is absent. Tools do not return recovery guidance. E.g., if ask fails due to insufficient context, no message suggests retrying with smaller top_k or broader file_glob. Agents have no actionable next steps on failure.
ask and search tools accept optional source enum=['holus'] but do not document what 'holus' means or how it differs from default behavior. LLMs cannot reason about when to use this filter.