Loading ToolBench
An MCP server that provides OSV (Open Source Vulnerabilities) database vulnerability information and querying capabilities
Three well-defined read-only tools with complete JSON schemas and proper input validation logic. Tool names follow verb_noun convention (query_*, get_*). All tools have descriptions and proper error handling with validation hints. Parameters are well-described with constraints documented in code. However, output schemas are not formally documented, and parameter descriptions could be more detailed about expected formats and constraints. Tool annotations are properly applied (readOnlyHint, idempotentHint, destructiveHint). Average tool score: 72/100.
Get details for a specific vulnerability by ID
Query for vulnerabilities affecting multiple packages or commits at once
Query for vulnerabilities affecting a specific package version or commit
Output schemas are not documented. The tools return vulnerability data, but agents have no formal contract for the response structure (fields, types, nesting). This violates the 'Document the output schema' rule and prevents agents from planning downstream tool calls or extracting structured data efficiently.
Parameter descriptions lack prescriptive format and constraint details. For example, 'commit' is described as 'The commit hash to query for' but does not specify expected length, encoding (hex?), or examples. The 'ecosystem' param lists examples (PyPI, npm, Go) but does not provide an enum or exhaustive list. This invites LLM hallucination of unsupported values.
Batch tool (query_vulnerabilities_batch) lacks pagination and result-limiting guidance. If an agent queries 1000 packages at once, there is no limit or indication of how results are structured (success/failure per item, or blanket result?). This risks context window exhaustion and prevents agents from handling partial failures gracefully.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | B | 73 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Mutual exclusivity constraints are validated in handler code but not clearly documented in parameter descriptions. Agents must infer from descriptions that 'commit' and 'version' cannot both be set, or that 'purl' excludes 'package_name'/'ecosystem'. Explicit upfront documentation in parameter descriptions (e.g., 'Cannot be set together with: version') would prevent incorrect agent attempts.
Error handling is present in code but error messages are generic. 'Failed to query OSV API' and 'Failed to marshal response' do not tell agents what to do next. Per the 'recovery-guide' pattern, errors should hint at remediation (e.g., 'Invalid ecosystem. Supported values: PyPI, npm, Go. For custom ecosystems, use purl parameter.').