Loading ToolBench
MCP server for financial risk analysis and document search using RAG (Retrieval-Augmented Generation). Integrates with FastAPI backend, MongoDB, Redis, and LangChain for AI-powered risk document analysis.
Single tool 'semantic_search' has a minimal description (17 chars) and incomplete schema documentation. The input schema shows two string parameters (query, doc_id) with basic descriptions, but the output is returned as a plain string with no structured schema defined. Error handling is present in the code but not documented in the tool definition. The tool name is action-oriented (semantic_search) which is a positive, but overall definition quality falls well below production standards.
Search for any specific context.
Tool description is critically short (17 characters: 'Search for any specific context.'). The rubric hard-rule requires descriptions under 20 chars to score 0-20. This violates DIMENSION 1.B critical check: 'Keep descriptions between 10 - 1024 characters' with context for LLM selection. Currently provides no guidance on WHEN to call this vs alternative search methods, prerequisites, or context about risk documents.
Output schema is undocumented. The tool returns a plain string (formatted markdown with source citations), but the tool definition declares no return schema. DIMENSION 1.D critical check requires: 'Document the output schema. LLMs need to know what fields to expect.' LLMs cannot reliably parse the formatted string response or extract page numbers and content programmatically without a declared schema.
Parameter 'doc_id' description lacks validation constraints. Description is 'The document ID to search within' with no format, length limits, or examples. DIMENSION 1.C critical check requires: 'Describe the expected format, range, and allowed values directly in the parameter description.' Without this, LLMs cannot reliably construct valid doc_id values and will make invalid calls.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 42 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 20 | - | v1 |
No error handling guidance in tool definition. The code catches exceptions and returns error strings, but the tool schema/description does not document what errors are possible, whether they are retryable, or what the LLM should do next. DIMENSION 1.E critical check requires: 'Error responses must tell the LLM what to do next.' The bare exception message 'Error during document retrieval: {str(e)}' will not guide agent recovery.
Parameter 'query' description is generic ('The search query to execute'). No guidance on query syntax, length limits, or special operators. This invites vague LLM queries that may not retrieve relevant risk documents. DIMENSION 1.B: 'Write descriptions as if prompt-engineering. State WHAT the tool does, WHEN to use it, and any prerequisites.'
No pagination or result limiting documented. The tool returns all retrieved_docs without limit or offset parameters. DIMENSION 1.D critical check: 'Tools returning lists should accept page/offset and limit parameters and return a total count or next_cursor. Without pagination, large results blow the context window.' Risk documents could be extensive; returning hundreds of matches will exhaust context.