Loading ToolBench
Single tool with minimal schema and weak descriptions. The 'scan' tool has a basic input schema but lacks output documentation, parameter descriptions, and error handling guidance. Tool name is reasonable but description is generic. This server demonstrates fundamental gaps in definition quality expected for production MCP tools.
Execute Trivy scanner to generate SPDX SBOM for a container image. Supports the SPDX JSON format.
Missing output schema documentation. The 'scan' tool returns SBOM JSON but provides no documented structure for what fields LLMs should expect. This prevents downstream tool composition and forces LLMs to guess at response structure.
Parameter 'image' lacks description in schema. While the tool docstring mentions it, the JSON Schema has no 'description' field for the 'image' parameter. LLMs cannot parse docstrings, they read schema descriptions only.
Tool description is generic (48 characters). Describes WHAT it does but not WHEN to use it, prerequisites, or what 'SPDX JSON format' means to an LLM. This is below the minimum actionable length.
No error handling guidance. Code returns 'Error: {str(e)}' on exceptions with no recovery hints. Per pattern:recovery-guide, error responses must tell the LLM what to do next, e.g., 'Trivy not installed, ensure it is in PATH' or 'Image not found, verify image name is correct.'
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 35 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 35 | - | v1 |
No input validation. The 'image' parameter accepts any string. Should validate non-empty, reasonable length (e.g., 1-255 chars), and document format constraints (e.g., 'docker.io/library/ubuntu:22.04' or 'myregistry.azurecr.io/myimage:latest').
No timeout or cancellation handling. Trivy scans can hang indefinitely on slow networks or unresponsive registries. No explicit timeout, no cancellation support, and no progress feedback. Per pattern:progress, tools calling external services should report progress or fail with clear timeout errors.
Hardcoded output file 'sbom.json' creates race conditions. If multiple concurrent scans run, they overwrite each other's output. Use unique temp files or return stdout directly.
Return type inconsistency. On success returns parsed JSON (dict), on error returns string. LLMs expect consistent types. Always return structured output, e.g., {success: bool, data: object, error: string}.
No pagination or result limits. If SBOM is very large (possible for complex images), returning entire JSON in one response can exhaust token budgets. Document maximum expected response size and offer chunking if needed.