Loading ToolBench
Model Context Protocol adapter for Calseta SOC data platform. Exposes security alerts, detection rules, context documents, workflows, and metrics as MCP resources and tools for AI agent consumption.
The server has 6 well-named tools with basic descriptions and consistent input schemas. However, descriptions are often too terse (20-40 chars, below the 194-char baseline), parameter descriptions are minimal, output schemas are undocumented, and error handling is absent. The tools follow verb_noun naming (post_alert, update_alert, execute_workflow, etc.), and input parameters have type definitions, but the overall quality falls short of production-grade. No tool annotations (readOnlyHint/destructiveHint), no pagination guidance for tools that might return large result sets, and no documented error recovery patterns. The server demonstrates competent basics but lacks the depth expected of A-grade tools.
Create a new detection rule
Enrich an indicator using configured enrichment providers
Execute a workflow with optional input parameters
Post a finding to an alert
Update the status of an alert
Update an existing detection rule
Tool descriptions are too brief (20-40 characters) compared to the 194-character baseline. 'Post a finding to an alert' and 'Update the status of an alert' lack context about when to use them, what they return, and their side effects.
Output schemas are not documented. The server does not specify what fields the tools return, their types, or whether they include pagination metadata, IDs for downstream chaining, or timestamps. LLMs cannot plan multi-step workflows without knowing the response structure.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) provided. The 'Risk' field in the evaluation indicates WRITE vs READ_ONLY, but the MCP protocol-level annotations are absent, preventing clients from making informed decisions about retryability and idempotency.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 57 | <=2025-11-25 | v2 |
| 2026-04-21 | F | 26 | - | v1 |
Parameter descriptions in the schema lack guidance on valid formats, constraints, and examples. For example, 'status' in update_alert_status accepts '(Open, Triaging, Escalated, Closed)' per the description, but these should be declared as an enum in the JSON Schema for machine-readability.
Error handling and recovery guidance are absent. No indication of what errors each tool can raise, whether they are retryable, or what the LLM should do next (e.g., 'If alert_uuid not found, call list_alerts() first').
The 'inputs' parameter in execute_workflow is of type 'object' with a generic description 'Optional input parameters for the workflow'. No schema for the nested object structure, LLMs cannot determine valid keys, their types, or constraints.
The 'providers' parameter in enrich_indicator is an array with a generic description but no guidance on valid provider names, allowed count, or what happens if an invalid provider is passed.
No dry-run or confirmation pattern for destructive operations (update_alert_status, create_detection_rule, update_detection_rule). Agents cannot safely preview changes before committing.