Loading ToolBench
Hex.pm MCP — package registry for the Elixir & Erlang ecosystems.
Four well-named, read-only tools with complete input schemas and clear descriptions. All tools follow verb_noun naming (get_*, search_*, list_). Descriptions are concise (100 - 180 chars) and explain what each tool does. Input schemas are properly typed with required fields and enums where appropriate (sort parameter in search_packages). However, output schemas are not documented, LLMs cannot see what fields to expect from responses. Error handling is minimal: hexGet() returns {error, message} on HTTP failures, but no recovery guidance or actionable next steps. Parameter descriptions lack format constraints (e.g., version string format, page bounds). No tool annotations (readOnlyHint, idempotentHint) despite all being read-only and idempotent.
Look up a single Hex.pm package (Elixir/Erlang mix package) by name. Returns description, licenses, links, latest version, and download counts.
Get details for a specific version (release) of a Hex.pm package (Elixir/Erlang mix package): dependency requirements, docs availability, publisher, and downloads.
List all published versions (releases) of a Hex.pm package (Elixir/Erlang mix package), each with its version and publish date.
Search the Hex.pm registry for Elixir/Erlang mix packages by keyword. Returns a list of matching packages with descriptions, latest versions, and downloads.
Output schemas not documented. LLMs cannot see what fields (name, version, downloads, html_url, etc.) are returned by each tool, forcing them to guess downstream field names and risking broken tool chains.
Error responses lack recovery guidance. hexGet() returns {error, message} on HTTP failures, but does not tell the LLM what to do next (retry, ask user, try alternative tool). A 404 for a package should suggest 'Package not found. Try search_packages() to find similar names.'
No tool annotations despite all tools being read-only and idempotent. Adding readOnlyHint and idempotentHint helps LLMs understand safety and retry behavior without reading descriptions.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
Parameter constraints not documented in descriptions. 'page' parameter lacks bounds (e.g., 1 - 1000); 'version' lacks format hint (e.g., semver); 'sort' enum is present but not explained when to use each option.
search_packages returns unbounded results. No pagination limit stated in description; no total_count or next_cursor in response structure. Large result sets could exhaust context window.