Loading ToolBench
An MCP server that provides code search capabilities using Azure AI Search and Azure OpenAI embeddings
Three tools with basic descriptions and partial schemas. All tools have verb-noun naming (search_code, get_file_content, list_symbols_in_file) which is good. However, descriptions are minimal (50-80 chars, below the 194-char baseline), lack WHEN/WHY context, and do not explain prerequisites or error recovery. Input schemas present but lack detail: top_k has no min/max bounds, file_path lacks format guidance. No output schemas documented. No error handling guidance. Tools are read-only (good for safety) but lack idempotent/readonly annotations. Parameter descriptions are sparse.
Retrieves the full content of a specific file from the Azure AI Search index
Retrieves indexed functions and classes for a specific file from the Azure AI Search index
Performs hybrid code search using vector embeddings and keyword matching against an Azure AI Search index
Descriptions lack WHEN/WHY context and are below baseline (40-80 chars vs 194-char average). LLMs cannot determine when to select search_code vs get_file_content without richer guidance.
No output schemas documented. LLMs cannot plan downstream calls or extract required fields (e.g., does search_code return file paths for get_file_content chaining?).
Parameter top_k lacks bounds (min/max). Unbounded integers let LLMs pass absurd values (e.g., top_k=999999) that may timeout or exhaust resources.
No error handling guidance. If Azure AI Search fails or file_path is invalid, LLMs receive raw errors with no recovery hints (e.g., 'Try search_code() first to find valid file paths').
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | <=2025-11-25 | v2 |
No tool annotations (readOnlyHint, idempotentHint). All three tools are read-only and idempotent, but this is not declared in the schema, forcing LLMs to infer safety.