Loading ToolBench
MCP server for discovering Azure SDK dependencies and retrieving code samples for Azure services
This server has two tools with reasonable names and brief descriptions, but suffers from critical gaps in schema documentation, parameter descriptions, and error handling. The code shows tool registration via McpServerTool attributes (good), but the input schemas visible in the evaluation show only minimal parameter definitions. Most critically, the parameter descriptions are extremely sparse, the 'path' parameter in both tools has the same one-line description without guidance on format, constraints, or expected behavior. The tool descriptions, while present, lack actionable guidance on WHEN to use each tool, what output format to expect, or how to handle errors. No output schemas are documented. The dependencies tool returns IEnumerable<string> with no indication of structure or format. The samples tool accepts a 'dependency' parameter but provides no enum or guidance on what valid values are. Error handling is absent, no recovery guidance, no categorization of error types, no actionable error messages. This is a domain-specific tool with good intent but execution falls well short of production quality.
Discover Azure SDK dependencies in your project. Identifies which Azure services (Key Vault, Storage, CosmosDB, Identity, etc.) your project uses. Supports Cargo.toml, .csproj, requirements.txt, pyproject.toml, go.mod, pom.xml, and package.json files.
Find code examples and documentation for Azure SDK libraries in your project. Retrieves README files and sample code for services like Azure Key Vault, Azure Storage, Azure Identity, CosmosDB, and more. Use this to quickly learn how to use the Azure services your project depends on.
No output schemas documented. Both tools return unstructured or minimally structured results (IEnumerable<string>, Task<T>) without documenting what fields, structure, or format the LLM should expect. This violates pattern:tool and makes it impossible for the LLM to compose follow-up calls or extract relevant data.
Parameter descriptions are minimal and generic. The 'path' parameter in both tools has identical one-line description 'The path where to start looking for project or manifest files' but provides no guidance on: format (absolute vs relative), constraints (must exist?), handling of nonexistent paths, or expected directory structure. The 'dependency' parameter in samples has no description of valid values, format, or how to find a valid dependency name.
No enumeration of valid dependency names or format guidance for the 'dependency' parameter in samples tool. LLMs will hallucinate invalid dependency names. A constraint or hint like 'Azure SDK package names matching pattern @azure/* or azure-* (e.g., @azure/identity, azure-storage-blob)' would be actionable.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 42 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
No error handling or recovery guidance. The code shows potential failures (e.g., invalid path, dependency not found, provider detection failure) but no documented error responses. LLMs have no guidance on what to do if dependencies() returns empty or samples() returns nothing.
Tool descriptions lack clarity on WHEN to use each tool and what the result enables. 'Discover Azure SDK dependencies' is vague on output format, does it return a flat list of package names, or structured objects with versions and repos? Similarly, 'Find code examples' does not explain what structure is returned (URLs? code snippets? README text?).
No pagination or result limiting specified. If a large project directory has dozens of manifest files or hundreds of dependencies, tools may return enormous result sets. No documented limit, pagination parameters (offset/limit), or guidance on handling large results.