Loading ToolBench
Socket MCP server for scanning dependencies and analyzing packages for security, quality, and supply-chain risks
Socket MCP server demonstrates solid definition quality with well-structured schemas, clear descriptions, and comprehensive parameter documentation. All 7 tools have explicit registrations with input schemas and descriptions. Tool naming follows verb_noun conventions ('depscore', 'alerts', 'threat_feed', 'package_files', 'package_file_contents', 'package_file_grep'). Most parameter descriptions are detailed and actionable. However, some tools use generic names ('alerts', 'organizations') that could be more specific, and a few descriptions could better guide LLM selection. Output schemas are documented but could be more explicit in the tool descriptions themselves. Error handling is basic with no explicit recovery guidance in descriptions.
List the latest security alerts for a Socket organization with the `alerts` tool. Requires `org_slug` — call the `organizations` tool first if you don't have it. Supports filtering by severity, category, status, artifact type/name, alert type, and repo. Use this to surface supply-chain, vulnerability, quality, license, and maintenance issues across the org's monitored packages. Results are paginated — pass the previous response's `endCursor` as `cursor` to fetch the next page.
Get the dependency score of packages with the `depscore` tool from Socket. Use 'unknown' for version if not known. Use this tool to scan dependencies for their quality and security on existing code or when code is generated. Stop generating code and ask the user how to proceed when any of the scores are low. When checking dependencies, make sure to also check the imports in the code, not just the manifest files (pyproject.toml, package.json, etc).
List the Socket organizations the authenticated user belongs to with the `organizations` tool. Use this to discover the `org_slug` values needed by other org-scoped tools (e.g. `alerts`, `threat_feed`), or when the user asks which organizations they have access to.
Read a single file from a package using the `package_file_contents` tool from Socket. Pass the `hash` printed next to each entry in `package_files` output. Returns up to 1 MB of UTF-8 text; binary files return metadata only.
Tool naming could be more specific: 'alerts' and 'organizations' are generic. Recommend 'list_security_alerts' and 'list_organizations' to clarify that they are discovery/listing operations.
Output schemas are not explicitly documented in tool descriptions. Users/LLMs cannot see what fields to expect without reading code. Descriptions mention return types (e.g., 'Returns a tree of file paths') but lack formal schema documentation.
Error handling descriptions are absent. Tools do not document what errors can occur, whether they are retryable, or how to recover. For example, 'depscore' does not document behavior when an invalid ecosystem is passed.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | B | 71 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 37 | - | v1 |
Search a single file from a package for lines matching a JavaScript regular expression. Pass the `hash` printed next to each entry in `package_files` output. The file is fetched from Socket once per session and cached, so repeated greps on the same hash skip the network. Returns matching lines with line numbers (grep -n style); binary files are refused. Useful for locating a specific symbol, import, or string inside a dependency without dumping the whole file.
List the files published in a package using the `package_files` tool from Socket. Returns a tree of file paths, each with its size and blob hash, for any package on a supported ecosystem (npm, pypi, gem, cargo, maven, golang, nuget, chrome, openvsx). Use `ecosystem` to specify the registry (default npm). Pass `depname`, `version`, and optionally `artifactId` and `platform` for disambiguation. The output includes a blob `hash` for each file, which you can pass to `package_file_contents` or `package_file_grep` to read or search the file.
Look up items in the Socket organization threat feed with the `threat_feed` tool. Requires `org_slug` — call the `organizations` tool first if you don't have it. Returns recently flagged packages (malware, typosquats, obfuscated code, etc.) along with a `nextPageCursor` for pagination. Use `filter` to narrow the threat category (default `mal` for malware), `ecosystem` to scope to a registry, or `name`/`version` to look up a specific package. Pass the previous response's cursor as `cursor` to fetch the next page.
No permission or scope declarations. Tools do not document what API credentials or permissions are required. The 'organizations' tool implies org-level access but does not state scopes.
Parameter 'ecosystem' in multiple tools is described as 'Package ecosystem (PURL type)...' but enums or allowed values are mentioned only in the description text, not as a formal enum constraint. LLMs may hallucinate unlisted ecosystems.