Loading ToolBench
mcp-nix demonstrates good naming, clear parameter descriptions, and comprehensive tool coverage for Nix ecosystem queries. All 9 tools follow verb_noun naming conventions (search_, read_, list_, show_, find_, help_). Descriptions are present and detailed (avg ~150 chars), exceeding the 10-char minimum. Input schemas are fully defined with types and descriptions for all parameters. However, output schemas are not documented in the visible source, the tools return formatted strings rather than structured objects, which limits downstream tool composition and increases token waste. Error handling is present but inconsistent: some tools format errors (InvalidChannelError, InvalidProjectError) while others return raw exception messages. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are present, though all tools appear to be read-only. Parameter defaults are sensible (channel='unstable', version='', limit=20). The code uses proper Pydantic validation and error classification via custom exceptions.
Find the nixpkgs commit hash for a specific package version. Searches Nixhub for the commit hash that contains a specific version of a package. Useful for pinning packages to exact versions in flake.nix or other configurations.
Get detailed help and examples for a Nix standard library function. Returns the function signature, description, arguments, and usage examples. Use search_nix_stdlib first to find the function path.
List available versions for a Nix project. Returns available versions/channels that can be used with the version parameter in search_options and show_option_details.
Read the Nix source code for a package derivation. Fetches and returns the .nix file that defines a package. Use search_nixpkgs first if you don't know the exact package name.
Read the Nix source code for an option declaration. Fetches and returns the module file that declares an option. Use search_options or show_option_details first to find the option name. Note: nix-nomad options don't have readable declarations as they are auto-generated from Nomad HCL specifications.
No documented output schemas. All tools return plain strings (str) rather than structured JSON objects with typed fields. This prevents LLMs from extracting specific data for downstream tool composition and wastes tokens on verbose text formatting.
Inconsistent error handling. Some errors are formatted with _format_error() (InvalidChannelError, InvalidProjectError, FunctionNotFoundError) but others may bubble up raw. Error messages should ALWAYS guide recovery: 'User not found. Try search_users() with a partial name.' vs bare 404.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 61 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 52 | - | v1 |
Search for Nix standard library functions. Searches the Nix standard library (lib.*) functions by name or description. Use this to find utility functions like strings, lists, attrsets operations.
Search for Nixpkgs packages by name or description. Returns package names, versions, and descriptions. For full details (homepage, license), use show_nixpkgs_package with the exact package name.
Search configuration options for a Nix project. Searches NixOS, Home Manager, NixVim, nix-darwin, impermanence, MicroVM, or nix-nomad options by name or description.
Get details for an option, or list all children if given a prefix. For leaf options like "services.nginx.enable", returns type, default, and description. For prefixes like "services.nginx", lists ALL child options exhaustively.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All 9 tools are read-only and idempotent, but this is not declared in the tool definitions. Without annotations, LLMs cannot infer which tools are safe to retry or parallelize.
Parameter format constraints documented in descriptions but not enforced at schema level. E.g. 'channel' accepts arbitrary strings; valid values (unstable, 24.11, 25.05) should be an enum. 'version' parameter in search_options and show_option_details allows empty string; constraint should be explicit.
No pagination support in search_nixpkgs. When result.total > len(result.items), the tool indicates there are more results but provides no way to fetch them. search_nix_stdlib has a limit parameter (good) but no offset/cursor for pagination.
String formatting in show_option_details response. The tool returns 'ALL child options exhaustively' as a formatted string, but LLMs need structured data to iterate, filter, and compose. Nested option hierarchies should return JSON arrays with typed fields.