Loading ToolBench
28 tools with consistent naming (verb_noun pattern) and comprehensive descriptions (avg 150-250 chars). All tools have input schemas with typed parameters and descriptions. However, output schemas are not documented in the provided code, and error handling lacks recovery guidance. Parameter descriptions are good but some lack explicit constraints (enums, ranges). No tool annotations (readOnlyHint/destructiveHint) visible. Dry-run patterns on write tools (restart_agent, run_active_response, add_to_cdb_list) are well-designed. Missing: output schema documentation, error classification, and actionable error messages.
Audit AR effectiveness — did blocks actually stop alert traffic from the source? For each AR event that blocked an IP, count alerts from that IP AFTER the block. Zero = block worked; non-zero = the attacker got through anyway.
Assign an agent to a group. Destructive — requires WAZUH_ALLOW_WRITES=true.
Add an IP, domain, or file hash to a CDB blocklist — takes effect immediately. Requires WAZUH_ALLOW_WRITES=true. list_name: e.g. 'malicious-ips' key: the IP/domain/hash to add value: label, e.g. 'c2-server', 'attacker', 'phishing'
Aggregated summary of alerts over a time window — counts by rule, agent, MITRE. Call this BEFORE search_alerts for broad questions like 'what happened today'. Returns aggregations only, not raw alerts — much smaller payload. Includes trend vs prior period and enriched MITRE technique names.
Date-histogram of alerts over time — spot spikes and quiet periods.
Output schemas not documented. Tools return data but LLMs cannot plan downstream calls without knowing response structure (e.g., what fields alert_summary returns, whether pagination is included).
Error handling lacks recovery guidance. No evidence of actionable error messages (e.g., 'User not found. Try search_users() first'). Errors should classify as retryable, user-fixable, or fatal.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 64 | <=2025-11-25 | v2 |
Check if Wazuh is silently dropping events due to queue pressure.
Drill into alerts mapped to one specific compliance control.
Aggregate alerts by compliance control for a given framework. framework: pci_dss | hipaa | gdpr | nist_800_53 | tsc Returns counts per control, plus top rules and top agents driving each.
Given a source IP or agent, return both the triggering alerts AND any AR taken.
Generate a compliance posture report for a given framework. Supported: pci_dss, hipaa, gdpr, nist_800_53, tsc. Returns control coverage, failing controls, and alert counts per control.
List active-response actions Wazuh took recently, with triggering context.
Get detailed info for a single agent by its ID (e.g. '001').
Get the full key:value contents of a CDB list file.
Full health check of Wazuh cluster nodes and the Indexer (OpenSearch) cluster.
List agents that belong to a given group.
List Wazuh agents filtered by status. status: active | disconnected | pending | never_connected
List all CDB lookup lists configured in Wazuh.
List Wazuh agent groups with their member counts and config status.
Show how many recent alerts came from this IP before adding it to a blocklist. Run this BEFORE add_to_cdb_list to understand noise-reduction impact. No writes performed — always safe.
Remove an entry from a CDB list. Requires WAZUH_ALLOW_WRITES=true.
Restart a Wazuh agent. dry_run=True (default) — shows what would happen without executing. Set dry_run=False to actually restart. Requires WAZUH_ALLOW_WRITES=true.
Trigger an active response command on an agent (e.g. firewall-drop). dry_run=True (default) — shows what would be sent without executing. Set dry_run=False to actually trigger. Requires WAZUH_ALLOW_WRITES=true.
Search Wazuh alerts in the Indexer. time_range: relative time like '15m', '1h', '24h', '7d' min_level: minimum rule level (default 7) agent_id: optional agent filter rule_groups: optional rule-group filter (e.g. ['authentication_failed', 'ssh'])
Search the full Wazuh archives index — all ingested logs, not just alerts. Use for forensic reconstruction when an attacker bypassed detection. query_string: Lucene syntax, e.g. 'data.srcip:198.51.100.42' Requires archiving enabled in ossec.conf.
Search the full archive index for a specific agent over a time range. Returns a chronological event timeline — useful for forensic investigation. All ingested logs, not just alerts. Requires archiving enabled in ossec.conf.
Find source IPs with repeated authentication failures (brute force candidates). Returns sources with more than `threshold` failures in the time range.
Find alerts mapped to a specific MITRE ATT&CK technique. technique_id: e.g. 'T1110' (brute force), 'T1059' (command interpreter)
Find all alerts originating from a specific source IP. Useful for IoC pivoting.
No tool annotations visible. Tools lack readOnlyHint, destructiveHint, or idempotentHint metadata required by current MCP spec (2026-07-28) for proper agent planning.
Parameter constraints not formalized. 'status' in list_agents has enum values in description text ('active | disconnected | pending | never_connected') but not as JSON Schema enum. Same for 'framework' in compliance tools. LLMs cannot parse text constraints reliably.
Pagination not explicit. Tools like search_alerts, search_by_mitre, search_archive_logs accept 'limit' but no 'offset', 'page', or 'next_cursor'. Large result sets risk context window exhaustion.