Loading ToolBench
Model Context Protocol server for Splunk, enabling AI assistants to query, analyze, and manage Splunk data and configurations
Two tools with minimal definitions. Tool names lack clear action verbs (SA-S4R_s4r_workshop_mode_get/set are prefixed with app namespace, not verb-first). Descriptions are present but generic (10-50 chars, below 34-char baseline for p10). First tool has no input schema visible. Second tool has basic enum constraint but lacks comprehensive parameter documentation. No output schemas documented. No error handling guidance. No composition patterns evident.
GET current Buttercup workshop Eventgen mode (infrastructure vs threat)
POST to set Buttercup workshop Eventgen mode (infrastructure or threat)
Tool names do not start with action verbs. 'SA-S4R_s4r_workshop_mode_get' and 'SA-S4R_s4r_workshop_mode_set' are namespace-prefixed; LLMs cannot infer intent from names alone. Should be 'get_workshop_mode' and 'set_workshop_mode'.
SA-S4R_s4r_workshop_mode_get has no visible input schema. GET endpoint should document any query parameters or constraints.
Descriptions are too short (10-50 chars vs 34-392 baseline). 'GET current Buttercup workshop Eventgen mode' lacks context on WHEN to use, WHAT it returns, and prerequisites. Should be 50-200 chars with actionable guidance.
No output schemas documented. LLMs cannot plan downstream calls or extract return values. GET should document response structure (e.g., {mode: string, timestamp: string}). SET should document success/failure response.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | 2026-07-28+ | v2 |
No error handling guidance. If mode is invalid or Eventgen is unavailable, what should the LLM do? No recovery hints, retryability classification, or actionable error messages defined.