Loading ToolBench
MCP server that integrates Splunk security event management with LLM capabilities for security analysis. Provides tools for Splunk search, index listing, AI-powered analysis via Ollama, and security event simulation.
This MCP server has critical definition quality gaps across all dimensions. While 5 tools are declared with basic descriptions, the schemas are minimal or absent, parameter descriptions are missing entirely, and there is no output schema documentation. The tool names follow a reasonable verb_noun pattern (splunk_test, splunk_search, ask_model), but parameter documentation is sparse. For example, splunk_search accepts 5 parameters (query, earliest_time, latest_time, count, add_ai) but only query and add_ai have descriptions; earliest_time, latest_time, and count lack any guidance on format, range, or defaults. No output schemas are documented, making it impossible for LLMs to understand what fields to expect or how to chain results into downstream calls. Error handling is not evident in the source code. The server appears to be a thin wrapper around Ollama and Splunk APIs without production-grade tool engineering.
Asks the Ollama AI model a question and returns the response
Analyzes Splunk logs using AI and provides security analysis summary
Lists all available indexes in the Splunk instance
Executes a Splunk search query and returns results. Can optionally include AI analysis of the results.
Tests connection to Splunk instance and returns version information
No output schemas documented for any tool. LLMs cannot infer what fields are returned, preventing proper result chaining and context planning.
Most parameters lack descriptions. splunk_search declares earliest_time, latest_time, and count with no guidance on format, range, or defaults. LLMs must guess valid inputs.
No input schema validation rules are defined. Parameters like 'count' lack min/max bounds; 'query' and 'splunk_results' lack length constraints or format guidance.
Tool descriptions are brief and lack actionable context. 'Executes a Splunk search query and returns results' does not explain when to use this vs ask_model, what prerequisites exist, or whether the operation is idempotent.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 35 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 26 | - | v1 |
No error handling guidance visible. No recovery hints, categorization of retryable vs fatal errors, or actionable error messages documented.
Potential composition issue: ask_model and get_ai_analysis appear to be alternative paths to AI analysis. Tool selection is ambiguous, when should the LLM choose ask_model vs get_ai_analysis?
Default values for splunk_search (earliest_time='0', latest_time='now', count=20) are mentioned in descriptions but not enforced in schemas. No MIN/MAX constraints declared.