Loading ToolBench
A read-only MCP server that queries a configurable metrics backend (list/describe/query/summary) as typed tools, with credentials resolved at call time and a schema-agnostic backend interface.
Server has 7 well-structured tools with complete JSON schemas and descriptions. Naming follows verb_noun convention (metrics_list, metrics_query, telemetry_emit_*). Descriptions are clear and actionable (avg ~150 chars). All parameters have types and descriptions. However, output schemas are not documented, LLMs cannot predict response structure. Error handling exists but lacks recovery guidance. No tool annotations (readOnlyHint/destructiveHint). Missing parameter constraints (e.g., 'agg' enum, 'limit' bounds). Composition is sound: tools chain well (list → describe → query). Security is good: no secrets in params, read-only tools properly marked.
Describe one source: its description and column -> type schema.
List the telemetry sources (datasets/tables/metrics) the backend exposes.
Run a bounded, read-only query against ``source`` over ``[start, end)``. ``start`` / ``end`` are ISO-8601 instants. ``filters`` is an optional column -> value mapping (values are parameter-bound by the backend, never string-interpolated). ``limit`` is capped by the core. Returns structured rows.
Return a single aggregate (``count``/``sum``/``avg``/``min``/``max``) of ``metric`` over ``[start, end)``.
Emit an occurrence marker as an event/log signal. If ``traceparent`` is supplied, the event is attached to that active trace. Without it, the event is still valid and no orphan span is created.
Emit an ad-hoc value as an OpenTelemetry metric.
Output schemas not documented. LLMs cannot predict response structure (e.g., what fields does metrics_query return? Is it {rows: [...], count: int}?). Forces agents to guess or fail on downstream processing.
metrics_summary 'agg' parameter lacks enum constraint. Description says 'count, sum, avg, min, or max' but no formal enum. LLMs may hallucinate invalid values like 'median' or 'stddev'.
metrics_query 'limit' parameter lacks bounds. Description says 'capped at 10000' but no min/max in schema. LLMs may pass negative or absurdly large values.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | <=2025-11-25 | v2 |
Emit a bounded operation span parented to ``traceparent``. A missing or invalid ``traceparent`` is rejected so the server never creates an orphan span.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Agents cannot infer which tools are safe to retry or have side effects. metrics_* are read-only; telemetry_emit_* are destructive.
Error messages lack recovery guidance. ToolError is raised but no hint on what to try next (e.g., 'Unknown source. Call metrics_list_sources() to see available sources.').