Loading ToolBench
A Netlify MCP server for searching and managing Netlify sites
The Netlify MCP server has solid tool naming and reasonable descriptions, but lacks comprehensive parameter descriptions, output schema documentation, and error handling guidance. All 4 tools are explicitly defined with schemas in main.py using FastMCP decorators. Tool names follow verb_noun conventions (search_sites, get_site_details, list_site_deploys, get_deploy_details), which is good. However, parameter descriptions are minimal or absent in the schema, and output schemas are not documented. Error handling is present (ValueError for missing token) but lacks recovery guidance. The server handles pagination correctly with page/per_page parameters, which is a positive pattern match.
Get detailed information about a specific deploy.
Get detailed information about a specific Netlify site.
List deploys for a specific Netlify site.
Search for Netlify sites by name.
Output schemas are not documented. Tools return dicts with multiple fields (e.g., get_site_details returns 15+ nested fields across urls, ssl, build_settings, published_deploy, settings objects), but the output structure is not declared in the tool registration. LLMs cannot plan downstream tool calls or extract specific fields when return schemas are undocumented.
Parameter descriptions are minimal. The schema shows 'name', 'page', 'per_page', 'site_id', 'deploy_id' parameters but only the high-level docstring in the function (e.g., 'Page number for pagination (default: 1)') documents them. The JSON Schema inputSchema passed to FastMCP must include explicit 'description' fields for every parameter; these appear to be inferred from function signatures rather than explicitly defined.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 57 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 50 | - | v1 |
Error handling lacks recovery guidance. When NETLIFY_API_TOKEN is not set, the tool raises ValueError('Netlify API token not configured. Check NETLIFY_API_TOKEN environment variable.'), which is slightly better than a raw error but does not tell the LLM what to do: should it retry? Ask the user? This is a server-initialization issue, not a call-time recovery path. pattern:recovery-guide requires error responses to tell the agent what to do next.
No HTTP error handling for API failures. Tools call make_netlify_request(), which calls response.raise_for_status(), but do not catch httpx.HTTPError or provide actionable feedback. A 401 (invalid token), 404 (site not found), or 429 (rate limited) will propagate as raw exceptions without guidance to the LLM.
search_sites accepts optional 'name' parameter but does not document fallback behavior clearly. The description says 'returns all sites if empty', but does not state: How many sites? Are they paginated? What is the default limit? NETLIFY_MAX_RESULTS is used (default 100), but this constraint is not visible in the tool description. An LLM calling search_sites('') might expect 1000 results and hit context limits.
list_site_deploys per_page parameter is capped at 100 with 'min(per_page, 100)' but this constraint is not documented in the description. An LLM might pass per_page=1000 expecting 1000 results; the silent clamping to 100 is non-obvious.