Loading ToolBench
Connects to the National Vulnerability Database (NVD) to retrieve CVE information, search vulnerabilities, and check code dependencies for security issues
Five tools with basic verb-noun naming (get_, search_, filter_, check_). All have descriptions (50-150 chars, within baseline 34-392 range). Input schemas present with type declarations. However, critical gaps: (1) output schemas completely undocumented, LLMs cannot plan downstream calls or extract structured data; (2) parameter descriptions lack constraints (e.g., 'limit' has no min/max bounds, 'severity' enum not enforced in schema); (3) no error handling guidance, tools return raw API errors or generic 'No details found' messages; (4) check_code_vulnerabilities has vague 'input_data' object schema with no field-level documentation; (5) no pagination support despite tools returning lists. Naming is clear but schemas are skeletal.
Checks for vulnerabilities in the provided list of dependencies.
Filters CVEs by their CVSS v3 severity level.
Retrieves detailed information for a specific CVE by its ID.
Fetches the most recent CVEs from the last specified number of days.
Searches for CVEs containing the specified keyword.
Output schemas completely undocumented. All tools return unstructured strings (e.g., 'CVE ID: ... Description: ...'). LLMs cannot parse structured fields, plan chained calls, or extract data reliably. Baseline: 100% of A+ tools document return types.
Parameter constraints missing from schema. 'limit' and 'days_back' have no min/max bounds; 'severity' accepts free-form strings instead of enum constraint. Baseline: constrained-input pattern requires enums for known value sets.
No error recovery guidance. Tools return raw API errors ('Failed to fetch CVE data: 429') or generic messages ('No details found'). LLMs cannot determine if error is retryable, user-fixable, or fatal. Baseline: recovery-guide pattern requires actionable error messages.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 44 | <=2025-11-25 | v2 |
check_code_vulnerabilities has opaque 'input_data' object parameter with no field-level schema or description. Requires nested 'dependencies' list with 'name' and 'version' fields, undocumented. LLMs cannot infer correct structure.
No pagination support. search_cves_by_keyword and filter_cves_by_severity accept 'limit' but return unstructured text with no total_count, next_cursor, or offset. Large result sets will blow context window. Baseline: paginated-result pattern required for list-returning tools.