Loading ToolBench
An MCP server that integrates Ollama LLM, MCP tools, and Splunk SIEM for security monitoring and LLM activity logging. Provides JSON-RPC MCP protocol support with tool execution, model chat, and Splunk HEC event ingestion.
This MCP server has severe definition quality issues. Only 2 tools are defined (chat, list_models), and both lack critical schema documentation, parameter descriptions, and proper error handling guidance. The 'chat' tool has a partial JSON Schema visible in seed-mcp-index.ps1, but the 'messages' parameter description is minimal ('Array of message objects with role and content'). The 'list_models' tool has an empty properties object with no schema detail. Neither tool has output schemas documented. Descriptions are present but generic (10-30 chars), falling well below the 50-200 char baseline for production tools. No parameter validation guidance, no error recovery hints, no idempotency markers. The source code samples (PowerShell scripts) show infrastructure setup and Splunk HEC event creation, not MCP tool implementations, actual Python MCP server code is not visible in the provided source, making tool definitions inferred rather than directly observable.
Chat with an Ollama model
List available Ollama models
Missing output schemas for both tools. 'chat' tool does not document the structure of its response (e.g., is it a string? an object with 'text' and 'tokens'? a stream?). 'list_models' has no documented return type.
'chat' tool parameter 'messages' has a generic description ('Array of message objects with role and content') that does not specify the structure of each message object. What are the allowed roles? What fields are required? This violates the pattern requirement that every parameter needs a detailed description.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 43 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 32 | - | v1 |
'chat' tool parameter 'model' has a minimal description ('The Ollama model identifier (e.g. 'llama3.2:latest')') that includes an example value in the description. LLMs tend to reuse example values literally.' Should replace with an enum or pattern constraint.
No error handling guidance. Neither tool documents what errors are possible, what they mean, or what the LLM should do next. E.g., if Ollama is unreachable, should the LLM retry? If a model doesn't exist, should it call list_models first?
Tool definitions appear in a PowerShell seed script (seed-mcp-index.ps1) rather than in visible Python MCP server code. Actual tool implementation is not provided in source, making definitions inferred rather than directly observable.
No parameter constraints or validation guidance. 'model' parameter could benefit from an enum of available Ollama models or a pattern. 'messages' has no documented format, required fields, or length limits.
'chat' tool description ('Chat with an Ollama model') is only 24 characters, well below the 50-200 char baseline for A+ tools. Does not explain when to use this tool vs alternatives, what the response contains, or dependencies on list_models.
'list_models' description ('List available Ollama models') is only 28 characters. Lacks context: when should an LLM call this? What structure does each model object have? Does it include metadata like context window or parameters?