Loading ToolBench
MCP server that exposes Snyk security scanning tools for code vulnerabilities (SAST), dependencies (SCA), package health, and breakability analysis
Snyk Studio MCP exhibits significant gaps in definition quality. While 9 tools are registered with names and basic descriptions, most descriptions are extremely terse (10-30 chars), parameter documentation is inconsistent, and input schemas lack depth. Tool names follow verb_noun convention (snyk_sca_scan, snyk_code_scan, snyk_auth), which is correct, but descriptions fail to explain WHEN to use each tool, what its output contains, or what side effects occur. Parameters like 'org', 'severity_threshold', and 'json' lack detailed constraints and LLM-friendly guidance. The server registers toolAnnotations (readOnlyHint/destructiveHint) which is positive for protocol readiness, but the core definitions remain below production baseline. Avg tool description length is ~25 chars (target: 50-200 for LLM clarity). Zero tools document their output schema structure. The schema completeness is limited, only snyk_sca_scan and snyk_code_scan show structured input schemas; others like snyk_auth, snyk_logout, snyk_trust, snyk_send_feedback, snyk_version provide empty {} inputs with no documentation of what they actually do or return.
Authenticate with Snyk using OAuth
Check for breakability issues in dependencies
Snyk Code SAST (Static Application Security Testing) vulnerability scan
Logout from Snyk authentication
Check the health and quality of a package
Snyk SCA (Software Composition Analysis) vulnerability scan for dependencies
Send feedback about Snyk tools
Descriptions critically short and generic. Tools like snyk_version (20 chars: 'Get Snyk CLI version information'), snyk_auth (0 chars implied), snyk_logout (20 chars) fail to explain purpose, when to call, prerequisites, or output format. Baseline: 50-200 chars for LLM clarity; these are 5-30 chars.
Empty input schemas for 5 tools (snyk_version, snyk_auth, snyk_logout, snyk_trust, snyk_send_feedback return {} with zero parameter documentation). Additionally, empty schemas fail to document what these tools actually do or what they accept (e.g., does snyk_auth take a token? An OAuth code? Nothing?). This makes the tools useless to an LLM.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 47 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Manage folder trust settings for scans
Get Snyk CLI version information
No output schema documentation for any tool. Rubric baseline: 100% of A+ tools have documented return types. snyk_sca_scan and snyk_code_scan presumably return vulnerability data, but the schema structure, field names, and types are nowhere in evidence. LLMs cannot plan downstream tool calls or extract data without knowing what fields to expect.
Parameter descriptions missing or insufficient. snyk_sca_scan accepts 'severity_threshold' with enum [low, medium, high, critical] but no description of what threshold means (minimum? maximum? exact match?). 'org' parameter has only 'Snyk organization ID', does this accept a UUID, slug, or name? Must the org exist first? Parameters 'prune_repeated_subdependencies' and 'fail_on' lack guidance on when LLMs should set them.
No dependency hints or chaining information. If snyk_sca_scan or snyk_code_scan returns a list of vulnerabilities with IDs, downstream tools (e.g., hypothetical snyk_get_remediation) would need those IDs. Current schema does not document what fields are returned, making tool composition impossible. Rubric: 'If create_ticket returns a ticket_id, the agent can immediately pass it to add_comment. Broken references force extra lookup calls.'
Error handling non-existent. No evidence of recovery guidance (e.g., 'If org not found, call snyk_list_orgs()'), error classification (retryable vs. fatal), or actionable error messages. Rubric: 'Error responses must tell the LLM what to do next.' A scan returning 'error: authentication failed' tells the LLM nothing; it should return 'Authentication failed. Call snyk_auth() first.'
Tool name ambiguity: snyk_trust, snyk_auth, snyk_logout, and snyk_send_feedback are vague. Does snyk_trust 'add trust', 'check trust', 'revoke trust'? Is it for trusting folders, domains, or publishers? Rubric: 'The tool name alone should convey what happens when called. update_ticket_status is clear; modify_ticket is vague.' Recommend: trust_folder, revoke_folder_trust, or add_trusted_folder.
No documentation of side effects. snyk_auth, snyk_logout, snyk_trust, and snyk_send_feedback are marked WRITE tools but descriptions do not state what data they modify, whether changes are permanent, or if they can be safely retried. Rubric: 'If the tool modifies state (creates, updates, deletes, sends), the description must say so. Agents need to know which calls are safe to retry and which have irreversible consequences.' This is critical for agent safety.
No result pagination or limits documented. If snyk_sca_scan returns thousands of vulnerabilities, context window bloat risks hallucination. Rubric: 'If the next likely action requires a team_id, channel_id, and message_id, the current response must return all three.' For snyk_sca_scan, must the response include remediationPaths? package_ids? severity distribution? Without schema, LLMs cannot optimize. Also: 'Even if the API allows returning thousands of items, cap results at a reasonable limit (e.g. 20-50) and offer pagination.'