Loading ToolBench
MCP server for read-only access to Loki, Prometheus, and Tempo APIs with code execution mode
Server exposes 3 tools with significant definition gaps. Tool descriptions are present but lack actionable detail for LLM selection. The 'search' and 'execute' tools accept arbitrary Python code as input, a powerful but risky pattern that bypasses schema validation. Parameter descriptions exist but are minimal. No output schemas documented. No error handling guidance. The 'instances' tool has an enum constraint but lacks clarity on side effects. Overall, definitions are below production baseline (194 char avg description, 100% param coverage expected).
Execute Python code against LGTM backends (Loki logs, Prometheus metrics, Tempo traces). First use 'search' to look up query syntax, then write code using the available clients. Available in your code: loki (LokiClient): await loki.query_range(query, start, end, limit=100, direction="backward") -> LokiQueryResponse # .get_log_entries() -> list[dict] with keys: timestamp, line, labels await loki.instant_query(query, time=None, limit=100) -> LokiQueryResponse # For metric-type LogQL: count_over_time, rate, etc. # .data.result -> list of results with .model_dump() await loki.get_labels(start=None, end=None, query=None) -> list[str] await loki.get_label_values(label, start=None, end=None, query=None) -> list[str] await loki.get_series(match=['{app="x"}'], start=None, end=None) -> list[dict] await loki.get_patterns(query, start, end) -> dict # Detect log patterns await loki.get_index_stats(query, start, end) -> dict # streams, chunks, entries, bytes await loki.get_volume(query, start, end, limit=100) -> dict # Volume by labels prometheus (PrometheusClient): await prometheus.instant_query(query, time=None) -> PrometheusQueryResponse # .get_instant_values() -> list[dict] with metric labels and value await prometheus.range_query(query, start, end, step="1m") -> PrometheusQueryResponse # .get_range_values() -> list[dict] with metric labels and values timeseries await prometheus.get_label_names(match=None, start=None, end=None) -> list[str] await prometheus.get_label_values(label, match=None, start=None, end=None) -> list[str] await prometheus.get_series(match=['{job="x"}'], start=None, end=None) -> list[dict] await prometheus.get_metric_names(match=None) -> list[str] await prometheus.get_metadata(metric=None) -> dict tempo (TempoClient): await tempo.get_trace(trace_id, start=None, end=None) -> dict # OTLP JSON await tempo.search(query=None, tags=None, min_duration=None, max_duration=None, limit=20, start=None, end=None) -> TempoSearchResponse # .traces -> list with traceID, rootServiceName, rootTraceName, durationMs, spanCount await tempo.get_tag_names(scope=None, start=None, end=None) -> list[str] await tempo.get_tag_values(tag, start=None, end=None) -> list[str] await tempo.get_services(start=None, end=None) -> list[str] Helper functions: summarize_trace(trace_data) -> dict # Summary with stats, span tree from OTLP data Your code must be the body of an async function that returns the result. All client calls are async — use await. Times: Unix epoch seconds (str) or RFC3339. Durations: "100ms", "1s", "5m", "1h". Examples: # Discover available labels in Loki labels = await loki.get_labels() return labels # Query error logs resp = await loki.query_range('{app="myapp"} |= "error"', "1710200000", "1710300000", limit=20) entries = resp.get_log_entries()[:20] return [{"line": e["line"][:200], "ts": e["timestamp"]} for e in entries] # Error rate with Prometheus resp = await prometheus.instant_query('sum(rate(http_requests_total{status=~"5..."}[5m]))') return resp.get_instant_values() # Range query resp = await prometheus.range_query('rate(http_requests_total[5m])', "1710200000", "1710300000", step="1m") return resp.get_range_values() # Search slow traces resp = await tempo.search(query='{resource.service.name="api"}', min_duration="1s", limit=10) return [{"id": t.traceID, "service": t.rootServiceName, "duration_ms": t.durationMs} for t in resp.traces] # Get trace summary trace_data = await tempo.get_trace("abc123def456") return summarize_trace(trace_data)
No output schema documented for any tool. LLMs cannot predict return structure, forcing them to guess at downstream field names and types. 'search' and 'execute' return arbitrary Python values with no type hint.
'search' and 'execute' accept arbitrary Python code strings as input. No schema validation, no input constraints, no injection guards. LLMs can pass malicious code; server must validate and sandbox execution.
Descriptions lack actionable guidance on WHEN to use each tool. 'search' vs 'execute' distinction is unclear, when should an LLM call search first? What does execute return if the code fails? No error recovery hints.
'instances' tool description does not state whether set_default modifies persistent config or session state. No mention of side effects or whether the change persists across server restarts.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 42 | 2026-07-28+ | v2 |
Manage LGTM instances: list available instances or set default. Args: action: "list" to show instances, "set_default" to change default name: Instance name (required for set_default)
Search LGTM query language references. Use to look up LogQL, PromQL, or TraceQL syntax. The code runs as an async Python function body with these available: reference: dict with keys "logql", "promql", "traceql" Each value is a markdown string with full syntax documentation including: - Selectors, filters, operators, functions, aggregations - Examples for common query patterns Example: # Find all PromQL aggregation operators lines = reference["promql"].split("\n") return [l for l in lines if "sum" in l.lower() or "avg" in l.lower()] # Get TraceQL structural operator docs return reference["traceql"] Your code must return a value. Write it as the body of an async function. Examples: # Look up how to filter by duration in TraceQL section = reference["traceql"] return [l for l in section.split("\n") if "duration" in l.lower()] # Get LogQL parser expressions return reference["logql"]
Parameter 'code' in search/execute has description 'Python code to execute...' but no constraints on length, complexity, or allowed imports. No guidance on timeout, memory limits, or what happens if code hangs.