Loading ToolBench
MCP Server for EventSight - Claude-Powered Windows Event Log Analysis
EventSight MCP provides 8 tools with basic parameter documentation but significant gaps in schema clarity, error handling guidance, and output structure documentation. Most tools have descriptions, but they lack depth regarding when to use them and what the LLM should expect. Parameters are described but many lack type clarity in the visible schema. Output schemas are not formally documented, the code converts internal objects to dicts but doesn't specify the structure for downstream tool composition. Error handling is minimal (e.g., add_feedback returns an error dict on invalid verdict, but most tools lack guidance). The tool composition is reasonable (analysis → feedback → learning management), but chaining could be clearer with explicit output field documentation.
Add a new correlation rule to detect attack patterns.
Provide feedback on a finding to create a learning.
Delete a learning.
Edit a learning's insight text or Event IDs.
Analyze a Windows EVTX file for security threats.
List all correlation rules.
List all stored learnings.
Output schemas not formally documented. Tools return dicts with inferred structures (e.g., evaluate_evtx returns 'findings' array with 'id', 'severity', etc., but this structure is not declared in the tool definition). LLMs cannot infer downstream field names, forcing them to guess or discover via trial. Breaks tool composition.
Missing limit/pagination parameters on list and search tools. list_learnings and search_learnings accept a 'limit' but no offset/cursor/page. Large result sets will blow context window. No documentation of default behavior or maximum results.
Insufficient error recovery guidance. add_feedback returns a basic error dict on invalid verdict, but no guidance on what the LLM should do next (e.g., 'try with one of the valid verdicts'). Other tools have no visible error handling at all (e.g., evaluate_evtx, edit_learning, delete_learning). Agents cannot self-correct.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 37 | - | v1 |
Search learnings by keyword or semantic similarity.
Destructive tool (delete_learning) lacks confirmation/dry-run support. No guard against accidental deletion. Tool description does not warn of irreversibility.
Parameter descriptions lack validation constraints. event_ids is documented as 'comma-separated' but no example or regex pattern. batch_size accepts integer but no min/max bounds. Learning/rule IDs are strings with no format hint. This invites malformed input from LLMs.
Tool descriptions too brief to guide LLM selection. 'List all stored learnings' (23 chars) and 'List all correlation rules' (25 chars) do not explain when to call these vs other tools or what structure they reveal.
Inferred vs explicit schema mismatch. Per code review, tools are registered via @mcp.tool() decorator, which suggests FastMCP auto-generates schemas from function signatures. However, the function return type hint is 'dict', not a Pydantic model or TypedDict with explicit field names. This means the schema is inferred, not explicit, and tool output shapes are not statically verifiable.
Missing scope/permission declarations. Tools modify state (add_feedback, edit_learning, delete_learning, add_correlation_rule) but do not declare required permissions (e.g., 'write:learning', 'admin:rules'). No audit trail setup visible. Agents cannot operate under least-privilege.
Verdict enum in add_feedback is validated at runtime but not exposed as a constrained enum parameter in the schema. The description lists valid values, but schema should declare 'enum: [false_positive, true_positive, benign, needs_context]' for automatic LLM validation.