Loading ToolBench
Advanced MCP server for security detections with Detection Engineering Intelligence, Knowledge Graph (Tribal Knowledge), Elicitation, and Resource Subscriptions
The server provides 31 tools (some duplicated) with consistent naming patterns (verb_noun: search, get, list) and enums for constrained parameters. However, quality is mixed: tool descriptions range from excellent (80-120 chars, clear intent) to minimal/generic. Input schemas are present but parameter descriptions vary significantly. Output schemas are undocumented. Many tools lack error handling guidance. The server follows the chat-data-model principle (accepting detection IDs and names) but duplicates tools across files (list_actors, compare_sources appear twice) suggesting composition issues. Several list_* tools accept free-form string parameters (story_name, detection_type, process_name, datasource) without enum constraints, inviting hallucinated values.
Alias for get_coverage_summary — returns the same comprehensive coverage report.
Compare detection coverage across multiple threat actors.
Compare detection coverage between different sources for a given technique or tactic.
Compare detection coverage between different sources for a given technique or tactic.
Generate a MITRE ATT&CK Navigator layer for visualization of technique coverage.
Analyze detection coverage against a MITRE ATT&CK threat actor/APT group. Shows covered and uncovered techniques.
Get intelligence on a threat actor including techniques, detections, and coverage assessment.
Duplicate tool definitions across web/lib/mcp/tools.ts and web/lib/ai/tools.ts: 'list_actors' and 'compare_sources' registered twice with slight parameter/description variations. LLMs cannot disambiguate duplicates and may select wrong implementation.
Free-form string parameters lack enum constraints: list_by_process (process_name), list_by_datasource (datasource), list_by_detection_type (detection_type), list_by_story (story_name) accept arbitrary strings. LLMs will hallucinate invalid process names, datasources, or story names, causing silent failures or empty results.
Output schemas undocumented. No visible schema documentation for tool return types (e.g., what fields does 'get_stats' return? What structure does 'get_coverage_summary' provide?). LLMs cannot plan downstream tool calls or extract required fields for chaining.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 62 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Get detailed threat actor profile with techniques, software, coverage analysis.
Get full details for a single detection by its ID (Sigma UUID or Splunk/Elastic slug).
Get overall MITRE ATT&CK coverage summary — total techniques, covered techniques, coverage %, breakdowns by source and tactic, weakest/strongest tactics.
Get the original YAML content for a detection (useful for copying the rule verbatim).
Get summary statistics about the indexed detection corpus (total detections, per-source counts, last sync).
Get a summary of detection coverage across all MITRE ATT&CK tactics.
Get detection coverage for a specific MITRE ATT&CK technique. Shows which sources have detections.
Full technique detail: all covering detections (paginated), actors using it, procedures, per-source breakdown.
Deep intelligence for a single MITRE technique ID: detections by source, actors using it, related sub-techniques, coverage gaps.
Identify detection gaps for a specific threat profile (ransomware, apt, initial-access, persistence, credential-access, defense-evasion). Returns prioritized uncovered techniques.
List known MITRE ATT&CK threat actors. Can search by name.
List known MITRE ATT&CK threat actors with optional search filter.
List detections with pagination. Prefer search() or list_by_* filters for targeted queries.
List detections mapped to a specific CVE ID.
List detections that use a specific data source.
List detections filtered by detection type (e.g., anomaly, evasion, hunting).
List detections that cover a specific MITRE ATT&CK technique ID.
List detections that monitor a specific process name.
List detections filtered by severity level.
List detections filtered by a specific source (sigma, splunk_escu, elastic, kql, sublime, crowdstrike_cql, jamf_protect).
List detections grouped by analytic story.
List detections for a specific MITRE ATT&CK tactic.
Full-text search across all detection rules (name, description, query). Supports multi-word AND queries. Returns up to 50 detections by default.
Search for security detections by keyword, technique ID, or description. Returns matching detection rules.
Alias tool 'analyze_coverage' duplicates functionality of 'get_coverage_summary' with weaker description (60 chars vs 80 chars). Redundant aliases waste agent reasoning and complicate tool selection.
No error handling guidance. Tool descriptions do not indicate what failures are possible (invalid CVE, nonexistent actor, empty search results) or what the agent should do next. Generic error responses will leave LLMs unable to recover.
Inconsistent parameter naming across similar tools: 'search' tool uses 'limit' param, but 'search_detections' uses 'limit' inconsistently (type is string, not number). Type inconsistency invites incorrect parameter passing.
Parameter type inconsistency in 'search_detections': 'limit' is declared as string but should be number (matches 'search' tool which uses number). String limits force type coercion and may cause unexpected behavior.
No pagination total_count or next_cursor returned. Tools default to limit=50 but provide no visibility into whether results were truncated or how many total matches exist. Agents cannot determine if a result set is complete.