Loading ToolBench
MCP server for cloud cost analysis and FinOps operations, providing tools for cost summarization, variance analysis, and anomaly detection on billing data
The server implements 3 well-structured FinOps tools with complete input schemas and type definitions. Naming follows verb_noun convention (cost_summary, cost_diff, anomaly_check). Descriptions are present and domain-relevant. However, there are significant gaps: output schemas are not formally documented; error handling is minimal; descriptions lack actionable guidance on when to use each tool vs alternatives; no parameter dependencies are documented; and critical missing fields for chaining (e.g., anomaly_check does not document what fields it returns for downstream use). The tools are read-only and low-risk, but the schema and composition issues prevent a higher score.
Detect daily spend anomalies and attribute likely service/account drivers.
Compare spend between two periods ("what changed?") by service/account/tag.
Summarize cloud spend for a period (total + top drivers + tag coverage).
Output schemas not formally documented. The code shows return dictionaries with nested structures (e.g., cost_summary returns 'period', 'filters', 'total_cost', 'top_services', 'tag_coverage'), but no JSON Schema or structured documentation visible for downstream tools to rely on. LLMs cannot plan follow-up calls without knowing what fields to expect.
Descriptions lack actionable guidance on tool selection and dependencies. 'Summarize cloud spend...' does not explain when to choose cost_summary vs cost_diff vs anomaly_check. No hints on prerequisite lookups or data dependencies between tools. This forces LLMs to guess which tool to call first.
Parameter filter dependencies undocumented. For cost_summary: 'provider', 'account', 'service', 'tag_env', 'tag_team' are all optional, but no description explains how they interact (AND vs OR logic). The _filter_df implementation shows AND logic, but an LLM reading only descriptions cannot infer this.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 52 | - | v1 |
anomaly_check descriptions do not explain algorithm or interpretation. z_threshold and window_days are numeric with defaults, but no guidance on what values mean or how to interpret results. An LLM cannot decide whether to increase z_threshold without understanding statistical significance.
cost_diff dimension parameter lacks enum constraint. The code validates dimension in {service, account, tag_env, tag_team}, but the input schema does not declare this as an enum. LLMs may pass invalid values like 'provider' or 'region', triggering a ValueError that the tool does not handle gracefully.
Error handling is minimal. No try/catch or recovery guidance. If BillingLoader.load() fails, the exception propagates without context. If date parsing fails, isoparse() raises a ParserError with no recovery hint. Agents cannot self-correct.
No idempotency guarantees documented. If an agent retries cost_summary with identical inputs, will it return the exact same result? The code loads fresh data each time, so idempotency depends on the underlying BillingLoader, which is not documented.
BillingLoader implementation not visible. The tools depend on self.loader.load().df, but the BillingLoader class is imported from billing_loader and not included in the source. Without seeing how data is loaded, populated, and cached, it's impossible to verify correctness of date filtering, timezone handling, or data freshness.