Loading ToolBench
A local MCP server that gives LLMs read access to Grafana Loki logs and Prometheus metrics
Four well-named tools with strong descriptions (avg 180 chars) and complete JSON Schema input definitions. All parameters are typed and described. Tool names follow verb_noun pattern (list_*, query_*). Descriptions include WHEN to call, WHAT to expect, and prerequisites. However, output schemas are not formally documented, responses are returned as JSON strings without declared structure. Error handling is minimal (generic error strings). No pagination support despite tools returning potentially large result sets. Missing tool annotations (readOnlyHint, idempotentHint).
List metric names available in Prometheus, each with its type (counter, gauge, histogram) and help text. Call this first to discover metric names and whether a metric is a counter (use rate()) before writing PromQL. Optionally filter names by substring.
List all service names available in Loki. Call this first to discover valid service names before querying logs.
Fetch log lines from a service. Results are newest first and capped by limit. Set the time range with lookback_minutes or with start and end; if you give none of them it defaults to the last 30 days. There is no maximum window, so pass start and end for anything longer. Optionally filter by level (such as error, warn, info, debug, crit, trace), search by text, or match a regex pattern. If Loki returns a parse error, retry without the level filter.
Run a PromQL query against Prometheus to reason about metrics such as CPU, memory, or request rates. Returns a series over time by default (each series summarised with min/avg/max/peak then its data points); set instant=true for current values. Set the time range with lookback_minutes or start and end; if you give none it defaults to the last 30 days. Discover metric names with list_metrics first.
Output schemas not documented. Tools return JSON strings without declared field structure. LLMs cannot plan downstream operations or extract specific fields reliably.
No pagination support. query_logs and query_metrics can return large result sets (capped by limit param) but no offset/cursor or total_count. Large responses risk context window exhaustion.
Error handling returns generic error strings (e.g., 'loki is not configured'). No recovery guidance or actionable next steps for LLMs.
Tool annotations missing. No readOnlyHint or idempotentHint declared, even though all four tools are read-only and safe to retry.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
Parameter 'level' in query_logs accepts free-form strings (error, warn, info, debug, crit, trace) but not declared as enum. LLMs may hallucinate invalid values.