Loading ToolBench
MCP server for medical query research with tools for web search, PDF loading, database management, and report generation
Four tools with complete input schemas and descriptions, but significant quality gaps. Tool names follow verb_noun convention (web_search, create_markdown_report, add_to_database, load_pdf). Descriptions exist but are inconsistent in depth, web_search and create_markdown_report are adequate (60-80 chars), but add_to_database and load_pdf lack clarity on when to use them vs. each other. Parameters have types and defaults, but descriptions are sparse or missing context. No output schemas documented. Error handling returns plain strings without recovery guidance. No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Composition issue: add_to_database and load_pdf both write to the same FAISS index, unclear when to use which. Average tool score: 52.
Add text chunks to the FAISS vector database with metadata (source, page, title)
Saves the final answer as a markdown file in data/reports/. Returns the file path.
Extract text from a PDF, chunk it, embed it, and add it to the FAISS index. Args: pdf_path (absolute or relative path to PDF file), source_name (label to use as source in chunk metadata, defaults to PDF filename). Returns summary string with chunk count and any errors.
Search the web using DuckDuckGo and return formatted results with titles, summaries, and URLs
No output schemas documented. Tools return plain strings; LLMs cannot parse structured results or chain downstream calls. E.g., web_search returns formatted text, not JSON with title/url/summary fields.
Composition ambiguity: add_to_database and load_pdf both write to FAISS index. No guidance on when to use each. LLM cannot determine if it should chunk text manually then call add_to_database, or call load_pdf directly.
Error handling returns plain strings without recovery guidance. E.g., 'Error: index not found. Run python -m app.index_builder first.' is actionable but inconsistent. Other errors like 'Search failed: {exc}' provide no next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
Parameter descriptions lack context. 'source' in add_to_database is described as 'Source label for the text' but does not explain what values are valid or how it relates to source_name in load_pdf.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). web_search is read-only; create_markdown_report, add_to_database, and load_pdf are destructive/write operations. LLM cannot infer safety properties.