Loading ToolBench
Retrieval-Augmented Generation (RAG) MCP server that provides tools for searching documents in Elasticsearch and retrieving full document content by ID. Designed to support document-based question answering with vector and keyword search capabilities.
The RAG MCP server provides three well-defined search and retrieval tools with proper parameter schemas and descriptions. All tools have clear verb-noun naming and documented input schemas with type information. However, output schemas are inferred from Pydantic models rather than explicitly documented in the tool registration, parameter descriptions lack depth and context about when/why to use each tool, and error handling guidance is minimal. The server implements useful functionality but misses opportunities to optimize for LLM reasoning and error recovery.
Get document content by document ID.
List all available Elasticsearch indices with their descriptions. You MUST use this tool to find the index to search in.
Search documents by keyword in title or content.
Output schemas are not explicitly documented in tool definitions. They are inferred from Pydantic models in source code (SearchResults, DocumentContent, IndexListResult), but LLMs cannot see these models. LLMs need to know: what fields will search() return? Will next_cursor always be present? This forces agents to guess or reason about API response structures.
Parameter descriptions lack actionable context. 'index' is described as 'Index to search in' but does not explain: Is it required? How do I find valid index names? (Answer: call list_elasticsearch_indices_tool first). Similarly, 'cursor' lacks guidance: How do I get the initial cursor? What format is it? The description mentions 'Opaque cursor for pagination' which is good, but omits: 'Pass null/omit on first call; use next_cursor from previous results for subsequent pages.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 6 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 53 | - | v1 |
Tool descriptions lack 'when to use' guidance, forcing LLMs to infer. The difference between search() and get_document_by_id() is not articulated. A description should say: 'search: Use when you have a keyword or concept. Returns best-matching documents with titles and highlights. get_document_by_id: Use when you have a specific document ID (from search results). Returns full document content.' Without this, agents may waste tokens trying both tools.
Error handling in tool descriptions is absent. The code shows that get_document_by_id_tool raises NotFoundError with message 'Document with id {document_id} not found in index {index}', but the tool description does not guide LLM recovery. A production tool description should say: 'If document not found: verify the document ID from search results. Try calling search() to find the correct ID.' This prevents agent dead-ends.
search() tool lacks documentation of output limits and pagination behavior. The code implements a fixed limit of 10 results per page (size=10 in tools.py, line ~55), but this is not stated in the tool description. LLMs need to know: 'Returns up to 10 results per call; use next_cursor to fetch additional pages.' Without this, agents may assume all results are returned in one call.
Highlight field structure is not documented. The search() response includes optional 'highlight' with field names like 'content', 'title', but the format (array of strings, HTML tags) is not explained. Tools.py shows highlight extraction prefers 'content_ja' → 'content_en' → 'content_ngram' → 'content', but this logic is not visible to LLMs.
The embedding/vector search feature is conditionally enabled (enable_embedding flag in config) but is not documented in the search() tool description. LLMs do not know if search uses keyword-only (BM25) or hybrid (keyword + vector). This is material to understanding search behavior (hybrid might find more semantically relevant results).