Loading ToolBench
MCP server for document similarity search using a vector database
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation and output structure. search_documents has a complete input schema with types and descriptions (query: string, top_k: integer with default, index_path: optional string). get_index_info similarly has documented parameters. However, both tools lack explicit output schema documentation, returns are described as 'Dict' with informal structure. Parameter descriptions are present but minimal (10-30 chars), below the 72-char baseline for A+ tools. No error handling guidance for LLMs (e.g., 'If search fails, try...'), no idempotency hints, and no indication of which operations are safe to retry. Tool names follow verb_noun pattern (search_, get_) which is good, but descriptions lack WHEN/WHY context needed for LLM selection.
Get information about the currently loaded vector database index
Search for similar documents based on a query string
Output schemas not documented. Both tools return Dict with informal structure (e.g., 'Dictionary containing search results'). LLMs cannot plan downstream calls or extract fields without knowing the response schema. Baseline: 100% of A+ tools have documented return types.
Parameter descriptions are too brief (10-30 chars vs 72-char baseline). 'The search query text' and 'Number of results to return' lack context on format, constraints, or when to use non-default values. LLMs cannot infer valid ranges or formats from minimal descriptions.
No error recovery guidance. Both tools return {'error': '...'} on failure, but do not tell the LLM what to do next. E.g., 'Index not found at {path}. Ensure the index has been created with create_faiss_index.py' would guide recovery.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | <=2025-11-25 | v2 |
No tool annotations (readOnlyHint, idempotentHint). Both tools are read-only and idempotent, but this is not declared. LLMs cannot infer safety properties and may avoid retrying on transient failures.
top_k parameter lacks range constraints. Default is 5, but no min/max documented. LLMs may pass absurd values (0, 10000) that break the FAISS retriever or exhaust memory. Baseline: numeric params should specify min/max.