Loading ToolBench
MCP server for Yandex Search API v2 with web and AI search capabilities
Two tools with reasonable naming (verb_noun pattern) but severely hampered by: (1) missing parameter descriptions in the exposed JSON schema, (2) vague tool descriptions that do not clearly distinguish use cases or explain WHEN to use each tool instead of the other, (3) no documented output schema structure, (4) no input validation error messages or recovery guidance, (5) exposed API configuration in detail.py (hardcoded URLs, timeout constants) without secure secret handling patterns. Tool names are clear ('ai_search_with_yazeka', 'web_search'), but the descriptions and input handling are below production standards.
Use this tool when the user needs to search online, the search query is complex, and a short summary of the web search results will make the answer easier to understand. Always call this tool if the user explicitely asks to search with yazeka.
Use this tool when the user needs to search online and the search query is simple.
Input parameter schema lacks descriptions. The JSON schema for both tools shows 'query' and 'search_region' properties but does NOT include descriptions for these fields in the schema definition. Per the rubric, every parameter must have a description explaining what it controls. The tool docstrings mention requirements, but the formal schema, which LLMs parse for parameter meaning, is incomplete.
Tool descriptions do not clearly distinguish use cases or explain WHEN to choose one tool over the other. Both descriptions mention search context (complex vs simple query), but the distinction is subjective. LLMs cannot reliably decide which tool to call based on these descriptions. Add explicit guidance: e.g. 'Use ai_search_with_yazeka when the user asks for a synthesis/summary of multiple sources. Use web_search for quick fact lookups or simple queries.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 39 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 38 | - | v1 |
No documented output schema. Both tools return JSON strings with 'response' + 'sources' (ai_search) or 'responses' array (web_search), but this structure is not formally documented. LLMs cannot plan downstream operations or extract fields reliably without a declared output schema. Add a return type description for each tool.
Error handling returns raw error messages without recovery guidance. validate_input_data() in detail.py returns a generic error string, but does not guide the LLM on what to fix or what to try next. Per the rubric, error responses must tell the LLM what to do: 'Invalid search_region: got "xx", must be "tr" or "en". Please retry with a valid region.'
API credentials and configuration exposed in code. detail.py contains hardcoded API URLs (GEN_SEARCH_URL, WEB_SEARCH_URL) and timeout constants, but the API key is missing from the visible code, likely injected via environment variable at runtime. However, there is no explicit pattern for secret injection (e.g., loading from env with validation, or using a secure config loader). Add a defensive pattern: load API key from env, validate it is non-empty, and document the expected variable name.
Parameter 'search_region' accepts only two enum values ('tr', 'en') but is not formally constrained in the schema. It is declared as type string with a description mentioning valid values, but no JSON Schema enum constraint is visible. This invites hallucinated region codes from the LLM. Use an enum constraint in the schema definition.
No validation or error categorization for API failures. The HTTP request in detail.py uses requests.raise_for_status(), but errors are not caught or classified as retryable vs fatal. If the Yandex API returns 429 (rate limit) or 503 (service unavailable), the tool throws an unhandled exception. Wrap API calls in try/except, categorize errors, and return actionable messages.
No pagination or result limits. The web_search tool returns all results from the XML response without limiting the array size. If the API returns 100+ results, the response bloats the context window. Per the rubric, cap results at 20-50 items and document the limit in the tool description.
Inconsistent output formats between tools. ai_search_with_yazeka returns {'response': ..., 'sources': [...]} while web_search returns {'responses': [...]} with nested 'data' and 'source' fields. This inconsistency forces the LLM to parse different structures. Standardize: both should return {results: [{content: ..., source: ...}], total: ...}.