Loading ToolBench
An MCP server providing knowledge base RAG retrieval, S3 document access, and integration with business analysis tools including Confluence search, Wikipedia lookup, and vector database retrieval.
This MCP server has 6 tools with READ_ONLY operations across knowledge bases (Qdrant), document storage (S3), and external APIs (Confluence, Wikipedia). While input schemas are present with type information, descriptions are present but terse, and parameter descriptions are sparse. The server lacks tool annotations (readOnlyHint present in risk field but not in MCP schema), output schemas are not documented, and error handling is minimal. Average tool naming is clear and verb-forward. However, parameter documentation is inconsistent, many lack descriptions explaining what they control or what values are valid. No pagination patterns visible despite tools that could return large result sets (Confluence, RAG queries). Output structure not formally documented for any tool.
Load the full text document stored in S3-compatible object storage.
Load part of a text document using an S3 byte-range request.
Retrieve chunks from a selected Qdrant collection.
Retrieve relevant pages from Confluence (internal knowledge base) using CQL (Confluence Query Language). Searches in page titles and content, returns up to 3 results with cleaned HTML content.
RAG with full retrieval metrics from Qdrant knowledge base. Generates embeddings for the query and retrieves relevant documents from a specified collection with optional metrics collection.
Search Wikipedia for encyclopedic definitions. Searches articles, selects the most relevant result, and returns the first 1500 characters of content.
No output schemas documented for any tool. LLMs cannot plan downstream tool calls or extract fields without knowing return structure.
Parameter descriptions are sparse or missing. 'query', 'collection_name', 's3_uri' lack actionable constraints (length, format, valid values, required vs optional behavior).
No numeric bounds on integer parameters. 'top_k' in query_knowledge_base and run_generic_rag_with_metrics lack min/max, LLMs may pass absurd values (10000000) causing timeouts or API errors.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 43 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 32 | - | v1 |
Tool naming includes vague verbs. 'run_generic_rag_with_metrics' uses 'generic' and 'run', these are ambiguous. Should be 'search_rag_with_metrics' or similar. 'run_*' appears in 3 tools and signals unclear intent.
No error handling guidance in any tool. Tools that call external APIs (Confluence, Wikipedia, S3, Qdrant) lack error messages, retry logic, or actionable recovery steps. LLM cannot self-correct on failure.
No pagination support. Confluence returns 'up to 3 results' hardcoded; Qdrant RAG returns top_k without cursor/offset for large datasets. Tools that could return variable result sets lack limit, offset, or next_cursor parameters.
Overlapping tool functionality without clear differentiation. query_knowledge_base and run_generic_rag_with_metrics both search Qdrant; the description does not explain when to use each. LLMs will waste reasoning cycles choosing between them.
No tool annotations (readOnlyHint, idempotentHint, etc.) in MCP schema. Risk field in metadata shows READ_ONLY but this is not exposed to the client via tool annotations. MCP clients cannot infer tool safety.
Hardcoded limits and environment variable defaults not documented. Confluence max_chars=2000, Wikipedia 1500 chars, top_k defaults to env VECTOR_AMOUNT_RAG, none explained to the LLM. LLMs see limit behavior as a surprise.