Loading ToolBench
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation, error handling, and output structure. prometheus_query_range has 4 typed parameters with descriptions, but lacks output schema documentation and error recovery guidance. prometheus_alert_rules has no input parameters and minimal context. Neither tool includes actionable error messages, validation constraints, or guidance for LLM recovery. Descriptions are present but generic (194-char baseline suggests room for improvement). No tool annotations (readOnlyHint, idempotentHint) despite both being read-only operations.
Query the Prometheus /rules API. Returns a list of alerting and recording rules that are currently loaded. In addition it returns the currently active alerts fired by the Prometheus instance of each alerting rule.
Query Prometheus /query_range API endpoint and return an image of a plot of the time series monitoring data. Use this tool whenever the user asks about the status of their compute infrastructure.
No output schema documentation. prometheus_query_range returns Image(path=...) but LLMs cannot infer the structure. prometheus_alert_rules returns dict[str, any] with no field documentation. LLMs need explicit output schemas to plan downstream actions.
Error handling lacks recovery guidance. Both tools return raw error strings (e.g., 'Query failed: 400 - ...') without actionable next steps. LLMs cannot determine if errors are retryable, user-fixable, or fatal.
Parameter descriptions lack format constraints. 'start_time' and 'end_time' accept strings but do not specify format (ISO 8601? Unix timestamp? Natural language?). 'step' interval format is undocumented. LLMs will guess and pass invalid values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
prometheus_alert_rules has no input parameters but description does not explain what it returns or when to call it. Output is 'dict[str, any]', completely opaque to LLMs. No guidance on structure, field names, or how to extract alert data.
No tool annotations despite both tools being read-only. FastMCP supports readOnlyHint and idempotentHint, these should be declared to help agents understand safety and retry semantics.