Loading ToolBench
MCP server for managing pfSense firewalls through AI assistants
The server defines 20 tools with mostly adequate naming and descriptions. All tools are explicitly registered via @mcp.tool() decorators in src/mcp_pfsense/server.py. Tool names follow verb_noun conventions (get_*, list_*, add_*, delete_, apply_*, restart_) which is good. However, there are significant gaps: (1) No input schemas are visible in the provided code, only function signatures with Python type hints. JSON Schema declarations required by MCP are not shown. (2) Parameter descriptions vary widely, some are detailed (e.g., add_firewall_rule has 9 params all described), others are minimal (e.g., get_system_status and get_interfaces have empty input objects with no descriptions). (3) Output schemas are completely undocumented, no description of what fields each tool returns. (4) Error handling is not documented, no indication of what errors can occur, how to retry, or recovery paths. (5) No tool annotations present (readOnlyHint, destructiveHint, idempotentHint) despite clear risk levels. Most parameters lack validation guidance (enums, ranges, formats). The tool set is logically organized and domain-appropriate for pfSense management, but the MCP definitions are incomplete.
Create a DHCP static mapping (IP reservation) for a MAC address. Staged until apply_changes('dhcp') is called or apply=true is passed.
Create a DNS host override entry in Unbound DNS Resolver. Staged until apply_changes('dns') is called or apply=true is passed.
Add a firewall rule. Type is 'pass', 'block', or 'reject'. The rule is staged (not active) until apply_changes('firewall') is called or apply=true is passed here.
Apply ALL staged changes of a subsystem ('firewall', 'dhcp' or 'dns'). Requires confirm=true. This reloads the subsystem, activating every pending change — including any a human staged in the pfSense WebGUI and has not reviewed yet.
Delete a DHCP static mapping. Requires confirm=true. `interface` is the mapping's DHCP server — the `parent_id` value returned by list_dhcp_static_mappings; `mapping_id` is its `id` there. Staged until apply_changes('dhcp') is called or apply=true is passed.
No JSON Schema definitions visible for tool inputs. Source shows only Python function signatures with type hints (e.g., 'interface: str | None = None'), but MCP requires explicit JSON Schema. Cannot verify parameter types, enums, constraints, or descriptions at the MCP protocol level.
No output schemas documented. LLMs cannot determine what fields to expect from tool responses. For example, get_system_status returns dict[str, Any], the caller has no contract about whether it includes 'version', 'uptime', 'temperature', etc. This breaks downstream tool chaining and forces LLMs to guess at field names.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | 2026-07-28+ | v2 |
Delete a DNS host override by ID. Requires confirm=true. Staged until apply_changes('dns') is called or apply=true is passed.
Delete a firewall rule by its ID (the `id` from list_firewall_rules). Requires confirm=true. Staged until apply_changes('firewall') is called or apply=true is passed.
Get ARP table showing connected devices (IP, MAC, interface).
Read-only view of recent firewall log entries. Each entry contains its ID and raw log text from pfrest. `limit` caps the number of entries (default 50). This tool only reads logs; it never writes to the firewall.
Get gateway status including latency, packet loss, and online state.
List all network interfaces with status and configuration.
Check whether a subsystem ('firewall', 'dhcp' or 'dns') has staged, unapplied changes.
Get pfSense system status including version, CPU, memory, uptime, and temperature.
List active DHCP leases showing IP, MAC, hostname, and lease times.
List DHCP static mappings (IP reservations), optionally filtered by interface. Each mapping carries `parent_id` (its DHCP server / interface) — pass that as `interface` to delete_dhcp_static_mapping.
List DNS Resolver host overrides (local DNS entries).
List firewall aliases (IP groups, port groups, URL lists).
List firewall rules, optionally filtered by interface.
List all services and their running status.
Restart a service by name. Requires confirm=true.
No tool annotations present. Tools have explicit risk levels (READ_ONLY, WRITE, DESTRUCTIVE, IRREVERSIBLE, REVERSIBLE) documented in the spec, but the code shows no readOnlyHint, destructiveHint, or idempotentHint annotations. This prevents clients from rendering appropriate UI warnings or applying rate limits.
Confirmation-required parameters (confirm=true) are present but no recovery guidance is documented. Tools like delete_firewall_rule require confirm=true but do not explain what happens on confirmation failure, whether the operation is idempotent, or how to undo if something goes wrong.
Minimal descriptions on read-only discovery tools. Tools like get_system_status, get_interfaces, list_firewall_aliases, get_gateway_status, get_arp_table, list_services have descriptions under 75 characters and lack context about WHEN to call them or what output structure to expect.
No validation guidance on enum parameters. 'type' parameter in add_firewall_rule must be one of 'pass', 'block', or 'reject', and 'ipprotocol' must be one of known values, but no enum constraint is visible. 'subsystem' in get_pending_changes/apply_changes is restricted to 'firewall', 'dhcp', 'dns' but no enum is declared.
No error handling or recovery guidance documented. No indication of what HTTP status codes, timeouts, or API errors might occur, what they mean, or how an LLM should respond (retry, ask user, abort).
Credentials likely exposed in environment. PfSenseConfig.from_env() loads config (presumably API URL, username, password) from environment variables. If credentials are passed as plaintext env vars, they may leak into agent traces and container logs. Verify secrets are injected securely and never logged.