Loading ToolBench
A mcp server for Vertex AI Search (Discovery Engine)
Single tool 'knowledge-search' has a basic description but lacks critical depth and parameter guidance. The tool name lacks a clear action verb and the schema is minimal. Input parameter 'query' has a description but no constraints, format guidance, or examples of valid/invalid inputs. No output schema is documented, LLMs cannot predict what structure to expect from the response. Error handling is present in the implementation but not surfaced in tool metadata or descriptions. The server provides no guidance on when to use this tool, what prerequisites exist (API keys, environment setup), or how to handle common failure modes. Missing: pagination for large result sets, output schema documentation, security warnings about sensitive queries, idempotency guarantees.
Returns the result of searching knowledge base
Tool name 'knowledge-search' lacks a clear action verb at the start. Use 'search_knowledge_base' or 'query_knowledge_base' to clearly signal the action and match LLM expectations for verb_noun naming convention.
Parameter 'query' has only a 2-word description ('Query to search'). No guidance on format, length, constraints, or examples. LLMs cannot infer whether to use natural language, keywords, boolean operators, or structured syntax. Add: expected length (e.g. 5-500 chars), format hints (natural language vs keywords), and examples of valid queries.
No output schema documented. Tool registration only shows the request schema. LLMs cannot predict the response structure (is it a string? An object with fields? A list?). The implementation returns {content: [{type: 'text', text: string}]}, but this is not visible in tool metadata. Document the expected response structure explicitly.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 36 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 25 | - | v1 |
Tool description 'Returns the result of searching knowledge base' is generic and does not explain when to use this tool vs other search tools, what a 'knowledge base' is, or what prerequisites are needed (credentials, engine configuration, language settings). Add context: 'Searches a Vertex AI Search (Discovery Engine) knowledge base and returns cited answers with optional related questions. Use this to find information from your configured knowledge corpus.'
No pagination support. The tool accepts a free-form query but does not expose page/offset/limit parameters. If a query matches thousands of documents, result handling is unclear, does it return all matches, or truncate silently? Without pagination controls, large datasets can blow the context window. Add optional 'limit' (1 - 100, default 10) and 'offset' parameters, and document the default behavior.
Error handling is implemented in the search() function but not surfaced in tool metadata. The tool description does not explain possible failure modes (missing credentials, invalid engine, API timeout) or what the LLM should do if search fails. Add error guidance: 'If the search fails, check that PROJECT, ENGINE, and LANGUAGE environment variables are set correctly and that your Vertex AI credentials are valid.'
No idempotency guarantee documented. The tool makes two API calls (getSessionInfo + getAnswer), which could have side effects or inconsistent behavior on retries. If the session expires mid-call, does retry from the beginning create duplicate sessions? Add: 'This tool is idempotent, retrying the same query will not create duplicate answers or side effects.'
No security guidance on sensitive queries. Users may ask questions containing PII, credentials, or confidential information. The tool description does not warn that queries are sent to a remote service or advise users to avoid sensitive data. Add: 'Do not use this tool to search for sensitive information like passwords or PII, queries are logged by Vertex AI Search.'
Language parameter is hardcoded in the tool but exposed as a server-level environment variable. LLMs cannot change the search language at runtime. If an agent needs to search in multiple languages, it must restart the server or be constrained to a single language. Consider exposing an optional 'language' parameter (default to env var if not provided) to allow per-query language selection.