Loading ToolBench
MCP server (stdio) wrapping the Sumo Logic Search Job API for agent-driven log triage
Strong foundation with 11 well-defined tools, comprehensive schemas, and detailed parameter descriptions. All tools have clear names starting with action verbs (sumo_*). Schemas are complete with types and descriptions for all parameters. Tool annotations (readOnlyHint, destructiveHint, idempotentHint) are properly applied. Main gaps: output schemas are not formally documented in the tool definitions themselves (only visible in implementation code), and error handling lacks recovery guidance for LLMs. Descriptions are solid (100-200 chars typical) but could be more prescriptive about when to use each tool vs. alternatives.
Delete a search job and free its slot
Analyze a scope and describe its field schema, severity detection, and recommended filters
Scan messages for errors/warnings and render a compact digest grouped by severity and signature
Export search results to a file (NDJSON or CSV format)
Compute ranked facet counts for specified dimensions across a scope
Retrieve detailed data and recent alerts for a specific monitor
Output schemas not formally documented in tool definitions. While implementation code shows structured returns (text content with type='text'), the tool registration does not declare return types or field structures. LLMs cannot plan downstream tool calls without knowing what fields to expect.
Error responses lack recovery guidance. Code shows fail() returns raw error messages without actionable next steps. E.g., 'no.records.not.an.aggregation.query' tells LLM nothing about what to do, should guide: 'This query is not an aggregation. Try sumo_search() with a raw query instead.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
List Sumo Logic monitors matching optional filters
Find new error/warning messages since a given timestamp in a scope
Execute a Sumo Logic search query and retrieve raw log messages
Trace a single request by its correlation ID across all scopes
Render a time-series trend of message counts, optionally grouped by a dimension
Tool descriptions lack 'when to use' guidance. sumo_error_digest vs sumo_search both scan logs, descriptions don't explain when to pick one over the other. Agents waste reasoning cycles on tool selection.
Parameter 'detail' enum values (summary, compact, full, raw) lack descriptions. LLMs cannot infer what 'compact' vs 'full' means without explicit guidance on token cost, field inclusion, or use case.
Time range parameters (last, from, to) accept multiple formats (relative '15m', ISO-8601, epoch ms) but descriptions don't clarify precedence or mutual exclusivity. LLMs may pass conflicting values.